Commit history

1. Commits on Feb 20, 2024

   • [SPARK-46858][PYTHON][PS][BUILD] Upgrade Pandas to 2.2.0

     Show description for 8e82887
     

     itholic

     authored and

     dongjoon-hyun

     committed
     8e82887View commit details

     Copy full SHA for 8e82887

     View code at this pointBrowse repository at this point

   • [SPARK-46820][PYTHON] Fix error message regression by restoring `new_msg`

     Show description for 9389c6b
     

     itholic

     authored and

     HyukjinKwon

     committed
     9389c6bView commit details

     Copy full SHA for 9389c6b

     View code at this pointBrowse repository at this point
2. Commits on Feb 19, 2024

   • [SPARK-44826][TEST][CONNECT] Re-enable `test_series_iloc_setitem`

     Show description for 9f582d6
     

     itholic

     authored and

     dongjoon-hyun

     committed
     9f582d6View commit details

     Copy full SHA for 9f582d6

     View code at this pointBrowse repository at this point

   • [SPARK-46812][SQL][PYTHON] Make mapInPandas / mapInArrow support ResourceProfile

     Show description for c4e4497
     

     wbo4958

     authored and

     WeichenXu123

     committed
     c4e4497View commit details

     Copy full SHA for c4e4497

     View code at this pointBrowse repository at this point
3. Commits on Feb 18, 2024

   • [SPARK-47082][SQL] Fix out-of-bounds error condition

     Show description for 63e8a14
     

     nchammas

     authored and

     HyukjinKwon

     committed
     63e8a14View commit details

     Copy full SHA for 63e8a14

     View code at this pointBrowse repository at this point
4. Commits on Feb 17, 2024

   • [SPARK-47057][PYTHON] Reeanble MyPy data test

     Show description for d423596
     HyukjinKwon

     committed
     d423596View commit details

     Copy full SHA for d423596

     View code at this pointBrowse repository at this point
5. Commits on Feb 16, 2024

   • [SPARK-47068][PYTHON][TESTS] Recover -1 and 0 case for spark.sql.execution.arrow.maxRecordsPerBatch

     Show description for 3bb762d
     HyukjinKwon

     committed
     3bb762dView commit details

     Copy full SHA for 3bb762d

     View code at this pointBrowse repository at this point

   • [SPARK-47055][PYTHON] Upgrade MyPy 1.8.0

     Show description for 7668eb5
     HyukjinKwon

     committed
     7668eb5View commit details

     Copy full SHA for 7668eb5

     View code at this pointBrowse repository at this point
6. Commits on Feb 15, 2024

   • [SPARK-46687][TESTS][PYTHON][FOLLOW-UP] Skip MemoryProfilerParityTests when codecov enabled

     Show description for d72efc0
     HyukjinKwon

     committed
     d72efc0View commit details

     Copy full SHA for d72efc0

     View code at this pointBrowse repository at this point
7. Commits on Feb 14, 2024

   • [SPARK-47014][PYTHON][CONNECT] Implement methods dumpPerfProfiles and dumpMemoryProfiles of SparkSession

     Show description for 4b9e9d7
     xinrong-meng

     committed
     4b9e9d7View commit details

     Copy full SHA for 4b9e9d7

     View code at this pointBrowse repository at this point
8. Commits on Feb 13, 2024

   • [SPARK-46944] Follow up to SPARK-46792 (ChannelBuilder refactoring): Fix minor typing oversight

     Show description for fb77606
     

     cdkrot

     authored and

     HyukjinKwon

     committed
     fb77606View commit details

     Copy full SHA for fb77606

     View code at this pointBrowse repository at this point
9. Commits on Feb 12, 2024

   • [SPARK-47027][PYTHON][TESTS] Use temporary directories for profiler test outputs

     Show description for 24a9d25
     

     ueshin

     authored and

     dongjoon-hyun

     committed
     24a9d25View commit details

     Copy full SHA for 24a9d25

     View code at this pointBrowse repository at this point
10. Commits on Feb 8, 2024

    • [SPARK-47002][PYTHON] Return better error message if UDTF 'analyze' method 'orderBy' field accidentally returns a list of strings

      Show description for 6569f15
      

      dtenedor

      authored and

      ueshin

      committed
      6569f15View commit details

      Copy full SHA for 6569f15

      View code at this pointBrowse repository at this point

    • [SPARK-46689][SPARK-46690][PYTHON][CONNECT] Support v2 profiling in group/cogroup applyInPandas/applyInArrow

      Show description for 1a66c8c
      xinrong-meng

      committed
      1a66c8cView commit details

      Copy full SHA for 1a66c8c

      View code at this pointBrowse repository at this point

    • [SPARK-47005][PYTHON][DOCS] Refine docstring of `asc_nulls_first/asc_nulls_last/desc_nulls_first/desc_nulls_last`

      Show description for 45956f7
      LuciferYang

      committed
      45956f7View commit details

      Copy full SHA for 45956f7

      View code at this pointBrowse repository at this point
11. Commits on Feb 7, 2024

    • [SPARK-46688][SPARK-46691][PYTHON][CONNECT] Support v2 profiling in aggregate Pandas UDFs

      Show description for 9bd0d7c
      

      xinrong-meng

      and

      ueshin

      committed
      9bd0d7cView commit details

      Copy full SHA for 9bd0d7c

      View code at this pointBrowse repository at this point

    • [SPARK-46966][PYTHON] Add UDTF API for 'analyze' method to indicate subset of input table columns to select

      Show description for 31f85e5
      

      dtenedor

      authored and

      ueshin

      committed
      31f85e5View commit details

      Copy full SHA for 31f85e5

      View code at this pointBrowse repository at this point

    • [SPARK-46922][CORE][SQL] Do not wrap runtime user-facing errors

      Show description for 5789316
      cloud-fan

      committed
      5789316View commit details

      Copy full SHA for 5789316

      View code at this pointBrowse repository at this point

    • [SPARK-46996][SQL] Allow AQE coalesce final stage in SQL cached plan

      Show description for becc04a
      

      liuzqt

      authored and

      cloud-fan

      committed
      becc04aView commit details

      Copy full SHA for becc04a

      View code at this pointBrowse repository at this point

    • [SPARK-46984][PYTHON] Remove pyspark.copy_func

      Show description for 0bed573
      HyukjinKwon

      committed
      0bed573View commit details

      Copy full SHA for 0bed573

      View code at this pointBrowse repository at this point
12. Commits on Feb 6, 2024

    • [SPARK-46976][PS] Implement `DataFrameGroupBy.corr`

      Show description for cfbf3c7
      zhengruifeng

      committed
      cfbf3c7View commit details

      Copy full SHA for cfbf3c7

      View code at this pointBrowse repository at this point
13. Commits on Feb 5, 2024

    • [SPARK-46978][PYTHON][DOCS] Refine docstring of `sum_distinct/array_agg/count_if`

      Show description for 2e917fb
      

      LuciferYang

      authored and

      dongjoon-hyun

      committed
      2e917fbView commit details

      Copy full SHA for 2e917fb

      View code at this pointBrowse repository at this point

    • [MINOR][DOCS] Fix various broken links and link anchors

      Show description for 07f1341
      

      nchammas

      and

      yaooqinn

      committed
      07f1341View commit details

      Copy full SHA for 07f1341

      View code at this pointBrowse repository at this point

    • [SPARK-46955][PS] Implement `Frame.to_stata`

      Show description for 30190ed
      zhengruifeng

      committed
      30190edView commit details

      Copy full SHA for 30190ed

      View code at this pointBrowse repository at this point
14. Commits on Feb 2, 2024

    • [SPARK-44815][CONNECT] Cache df.schema to avoid extra RPC

      Show description for 6f87fe2
      

      3 peoplecommitted
      6f87fe2View commit details

      Copy full SHA for 6f87fe2

      View code at this pointBrowse repository at this point

    • [SPARK-46936][PS] Implement `Frame.to_feather`

      Show description for 53cbaeb
      zhengruifeng

      committed
      53cbaebView commit details

      Copy full SHA for 53cbaeb

      View code at this pointBrowse repository at this point
15. Commits on Jan 31, 2024

    • [SPARK-46931][PS] Implement `{Frame, Series}.to_hdf`

      Show description for 88f121c
      

      zhengruifeng

      authored and

      dongjoon-hyun

      committed
      88f121cView commit details

      Copy full SHA for 88f121c

      View code at this pointBrowse repository at this point

    • [SPARK-46932] Clean up the imports in `pyspark.pandas.test_*`

      Show description for e95e820
      zhengruifeng

      committed
      e95e820View commit details

      Copy full SHA for e95e820

      View code at this pointBrowse repository at this point

    • [SPARK-46753][PYTHON][TESTS] Fix pypy3 python test

      Show description for 0871a6f
      

      panbingkun

      and

      HyukjinKwon

      committed
      0871a6fView commit details

      Copy full SHA for 0871a6f

      View code at this pointBrowse repository at this point

    • [SPARK-46926][PS] Add `convert_dtypes`, `infer_objects` and `set_axis` in fallback list

      Show description for 5d87ac6
      zhengruifeng

      committed
      5d87ac6View commit details

      Copy full SHA for 5d87ac6

      View code at this pointBrowse repository at this point

    • [SPARK-46927][PYTHON] Make `assertDataFrameEqual` work properly without PyArrow

      Show description for 2f917c3
      

      itholic

      authored and

      HyukjinKwon

      committed
      2f917c3View commit details

      Copy full SHA for 2f917c3

      View code at this pointBrowse repository at this point

    • [SPARK-46925][PYTHON][CONNECT] Add a warning that instructs to install memory_profiler for memory profiling

      Show description for 4f0a9df
      

      xinrong-meng

      and

      HyukjinKwon

      committed
      4f0a9dfView commit details

      Copy full SHA for 4f0a9df

      View code at this pointBrowse repository at this point
16. Commits on Jan 30, 2024

    • [SPARK-46916][PS][TESTS] Clean up `pyspark.pandas.tests.indexes.*`

      Show description for e3143c4
      

      zhengruifeng

      authored and

      dongjoon-hyun

      committed
      e3143c4View commit details

      Copy full SHA for e3143c4

      View code at this pointBrowse repository at this point
17. Commits on Jan 29, 2024

    • [SPARK-46687][PYTHON][CONNECT] Basic support of SparkSession-based memory profiler

      Show description for 528ac8b
      

      xinrong-meng

      authored and

      ueshin

      committed
      528ac8bView commit details

      Copy full SHA for 528ac8b

      View code at this pointBrowse repository at this point

    • [SPARK-46897][PYTHON][DOCS] Refine docstring of `bit_and/bit_or/bit_xor`

      Show description for 5056a17
      

      LuciferYang

      authored and

      dongjoon-hyun

      committed
      5056a17View commit details

      Copy full SHA for 5056a17

      View code at this pointBrowse repository at this point