Thank you all for the replies! To @Nicholas Chammas <nicholas.cham...@gmail.com> : Thanks for cleaning up the error terminology and documentation! I've merged the first PR and let's finish others before the 4.0 release. To @Dongjoon Hyun <dongjoon.h...@gmail.com> : Thanks for driving the ANSI on by default effort! Now the vote has passed, let's flip the config and finish the DataFrame error context feature before 4.0. To @Jungtaek Lim <kabhwan.opensou...@gmail.com> : Ack. We can treat the Streaming state store data source as completed for 4.0 then. To @Cheng Pan <cheng...@apache.org> : Yea we definitely should have a preview release. Let's collect more feedback on the ongoing projects and then we can propose a date for the preview release.
On Wed, Apr 17, 2024 at 1:22 PM Cheng Pan <pan3...@gmail.com> wrote: > will we have preview release for 4.0.0 like we did for 2.0.0 and 3.0.0? > > Thanks, > Cheng Pan > > > > On Apr 15, 2024, at 09:58, Jungtaek Lim <kabhwan.opensou...@gmail.com> > wrote: > > > > W.r.t. state data source - reader (SPARK-45511), there are several > follow-up tickets, but we don't plan to address them soon. The current > implementation is the final shape for Spark 4.0.0, unless there are demands > on the follow-up tickets. > > > > We may want to check the plan for transformWithState - my understanding > is that we want to release the feature to 4.0.0, but there are several > remaining works to be done. While the tentative timeline for releasing is > June 2024, what would be the tentative timeline for the RC cut? > > (cc. Anish to add more context on the plan for transformWithState) > > > > On Sat, Apr 13, 2024 at 3:15 AM Wenchen Fan <cloud0...@gmail.com> wrote: > > Hi all, > > > > It's close to the previously proposed 4.0.0 release date (June 2024), > and I think it's time to prepare for it and discuss the ongoing projects: > > • > > ANSI by default > > • Spark Connect GA > > • Structured Logging > > • Streaming state store data source > > • new data type VARIANT > > • STRING collation support > > • Spark k8s operator versioning > > Please help to add more items to this list that are missed here. I would > like to volunteer as the release manager for Apache Spark 4.0.0 if there is > no objection. Thank you all for the great work that fills Spark 4.0! > > > > Wenchen Fan > >
