Thank you for the detailed proposal with dates, Hyukjin. I guess it's aligned with Herman already, too. I'm fine if we have a pre-defined and predictably achievable schedule. :-)
We know the well-known goal of `Spark Connect` and I love to have. I've been monitoring the progress, but I'm not sure if the delivery schedule is feasible or not in the given status. At this point, I believe you and Herman's words because you are the leaders and contributors for that area. For the proposed schedule, the worst case in the community would be a future delay due to the risk of unknown personal schedules because Apache Spark is a community-driven project depending on the contributors' voluntarily passion and time. And, we know that it varies during the following Holiday seasons much. I also believe that you and Herman considered them already in the personal schedules. - 2024.12.20 ~ 2025.01.05 (Christmas and Happy New Year Holiday) - 2025.01.29 ~ 2025.02.03 (Chinese New Year) > 2024-01-15 Creating `branch-4.0` (allowing backporting new features) > 2024-02-01 Feature Freeze (allowing backporting bug fixes only) > 2024-02-15 Starting Apache Spark 4.0.0 RC1 Thank you again for the replies. Let's wait and collect more opinions and update Apache Spark Website for the tentative schedule. Dongjoon. On 2024/09/27 03:00:49 Hyukjin Kwon wrote: > I meant 2025 :-). > > On Fri, Sep 27, 2024 at 11:15 AM Hyukjin Kwon <gurwls...@apache.org> wrote: > > > We're basically working on making Scala Spark Connect ready. > > For example, I am working on having a parent class for both Spark Classic > > and Spark Connect so users would face less breaking changes, and they can > > run their application without changing anything. > > In addition, I am also working on sharing the same test base between Spark > > Classic and Spark Connect. > > For those, I think it might take a couple of months to stabilize. > > > > What about the below schedule? > > > > - 2024-01-15 Creating `branch-4.0` (allowing backporting new features) > > - 2024-02-01 Feature Freeze (allowing backporting bug fixes only) > > - 2024-02-15 Starting Apache Spark 4.0.0 RC1 > > > > > > On Fri, 27 Sept 2024 at 07:35, Dongjoon Hyun <dongjoon.h...@gmail.com> > > wrote: > > > >> Thank you for the reply, Herman. > >> > >> Given that December and January are on your schedule, > >> I'm not sure what date your proposal is. Could you elaborate more? > >> As we know, the community is less active during that Winter period. > >> > >> In addition, although I know that you are leading that area actively with > >> big refactorings, > >> it would be greatly appreciated if you could share a more concrete > >> progress status and > >> delivery plan (or milestone) for `Connect and Classic Scala interface` to > >> the community. > >> Specifically, we are curious about how much we achieved between > >> `preview1` and `preview2`, > >> and if it's going to be stabilized enough at that time frame. Let's see > >> what you have more. > >> > >> > We are working on unifying the Connect and Classic Scala interface > >> > >> Thank you again! > >> > >> Dongjoon. > >> > >> On Thu, Sep 26, 2024 at 12:23 PM Herman van Hovell <her...@databricks.com> > >> wrote: > >> > >>> Hi, > >>> > >>> Can we push back the dates by at least 2 months? > >>> > >>> We are working on unifying the Connect and Classic Scala interface, and > >>> I would like to avoid rushing things. > >>> > >>> Kind regards, > >>> Herman > >>> > >>> On Thu, Sep 26, 2024 at 3:19 PM Dongjoon Hyun <dongjoon.h...@gmail.com> > >>> wrote: > >>> > >>>> Hi, All. > >>>> > >>>> We've delivered two preview releases for Apache Spark 4.0 successfully. > >>>> I believe it's time to discuss cutting branch-4.0 to stabilize more > >>>> based on them > >>>> and schedule feature freeze. > >>>> > >>>> - https://spark.apache.org/news/spark-4.0.0-preview1.html (June) > >>>> - https://spark.apache.org/news/spark-4.0.0-preview2.html (September) > >>>> > >>>> I'd like to propose as a candidate. > >>>> > >>>> - 2024-10-01 Creating `branch-4.0` (allowing backporting new features) > >>>> - 2024-10-15 Feature Freeze (allowing backporting bug fixes only) > >>>> - 2024-11-01 Starting Apache Spark 4.0.0 RC1 > >>>> > >>>> WDYT? Please let me know if you have release blockers for Spark 4 or > >>>> other schedule candidates in your mind. > >>>> > >>>> Thanks, > >>>> Dongjoon. > >>>> > >>> > --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
