Ideally, we should complete these - [SPARK-37093] Inline type hints python/pyspark/streaming - [SPARK-37395] Inline type hint files for files in python/pyspark/ml - [SPARK-37396] Inline type hint files for files in python/pyspark/mllib
All tasks have either PR in progress or someone working on a one, so the the limiting factor is our ability to review these. On 3/3/22 19:44, Maxim Gekk wrote: > Hello All, > > I would like to bring on the table the theme about the new Spark release > 3.3. According to the public schedule at > https://spark.apache.org/versioning-policy.html > <https://spark.apache.org/versioning-policy.html>, we planned to start > the code freeze and release branch cut on March 15th, 2022. Since this > date is coming soon, I would like to take your attention on the topic > and gather objections that you might have. > > Bellow is the list of ongoing and active SPIPs: > > Spark SQL: > - [SPARK-31357] DataSourceV2: Catalog API for view metadata > - [SPARK-35801] Row-level operations in Data Source V2 > - [SPARK-37166] Storage Partitioned Join > > Spark Core: > - [SPARK-20624] Add better handling for node shutdown > - [SPARK-25299] Use remote storage for persisting shuffle data > > PySpark: > - [SPARK-26413] RDD Arrow Support in Spark Core and PySpark > > Kubernetes: > - [SPARK-36057] Support Customized Kubernetes Schedulers > > Probably, we should finish if there are any remaining works for Spark > 3.3, and switch to QA mode, cut a branch and keep everything on track. I > would like to volunteer to help drive this process. > > Best regards, > Max Gekk -- Best regards, Maciej Szymkiewicz Web: https://zero323.net PGP: A30CEF0C31A501EC
OpenPGP_signature
Description: OpenPGP digital signature
