Re: [DISCUSS] Creating `branch-4.0` and Feature Freeze for Apache Spark 4.0

2024-09-26 Thread Hyukjin Kwon
I meant 2025 :-). On Fri, Sep 27, 2024 at 11:15 AM Hyukjin Kwon wrote: > We're basically working on making Scala Spark Connect ready. > For example, I am working on having a parent class for both Spark Classic > and Spark Connect so users would face less breaking changes, and they can > run thei

Re: [DISCUSS] Creating `branch-4.0` and Feature Freeze for Apache Spark 4.0

2024-09-26 Thread Hyukjin Kwon
We're basically working on making Scala Spark Connect ready. For example, I am working on having a parent class for both Spark Classic and Spark Connect so users would face less breaking changes, and they can run their application without changing anything. In addition, I am also working on sharing

Re: [DISCUSS] Creating `branch-4.0` and Feature Freeze for Apache Spark 4.0

2024-09-26 Thread Herman van Hovell
Hi, Can we push back the dates by at least 2 months? We are working on unifying the Connect and Classic Scala interface, and I would like to avoid rushing things. Kind regards, Herman On Thu, Sep 26, 2024 at 3:19 PM Dongjoon Hyun wrote: > Hi, All. > > We've delivered two preview releases for

Re: [DISCUSS] Creating `branch-4.0` and Feature Freeze for Apache Spark 4.0

2024-09-26 Thread Dongjoon Hyun
Thank you for the reply, Herman. Given that December and January are on your schedule, I'm not sure what date your proposal is. Could you elaborate more? As we know, the community is less active during that Winter period. In addition, although I know that you are leading that area actively with b

[DISCUSS] Creating `branch-3.5` and Feature Freeze for Apache Spark 4.0

2024-09-26 Thread Dongjoon Hyun
Hi, All. We've delivered two preview releases for Apache Spark 4.0 successfully. I believe it's time to discuss cutting branch-4.0 to stabilize more based on them and schedule feature freeze. - https://spark.apache.org/news/spark-4.0.0-preview1.html (June) - https://spark.apache.org/news/spark-4.

[DISCUSS] Creating `branch-4.0` and Feature Freeze for Apache Spark 4.0

2024-09-26 Thread Dongjoon Hyun
Hi, All. We've delivered two preview releases for Apache Spark 4.0 successfully. I believe it's time to discuss cutting branch-4.0 to stabilize more based on them and schedule feature freeze. - https://spark.apache.org/news/spark-4.0.0-preview1.html (June) - https://spark.apache.org/news/spark-4.

[ANNOUNCE] Announcing Apache Spark 4.0.0-preview2

2024-09-26 Thread Dongjoon Hyun
Hi, All. To enable wide-scale community testing of the upcoming Spark 4.0 release, the Apache Spark community has posted a Spark 4.0.0-preview2 release. This preview is not a stable release in terms of either API or functionality, but it is meant to give the community early access to try the code