As the release manager for 4.0, I’m awaiting a conclusion on this topic before cutting the next RC. There have been many debates, and I’m trying to understand what has happened so far. 1. A mistake was made, leading to a vendor name being included in the configuration released in Spark 3.5.4. 2. Dongjoon initiated a vote to deprecate the incorrect configuration name in 3.5.5, and the vote passed. Thanks to Dongjoon, 3.5.5 was released shortly after. 3. A PR <https://github.com/apache/spark/pull/49897> that simply renamed (rather than deprecated) the configuration was merged into master/4.0. This is a breaking change and was not backed by a vote. 4. This vote concerns adding migration logic to prevent the breaking change from affecting streaming queries.
Personally, I don’t think this vote is necessary, as the standard approach would be to revert the breaking change if there is disagreement. However, if we revert it, leaving the misnamed configuration in 4.0 is clearly not ideal. Following the previously voted decision for 3.5.5 and applying the same approach to 4.0.0 seems like a reasonable path forward. On Thu, Mar 13, 2025 at 7:43 PM Mark Hamstra <markhams...@gmail.com> wrote: > This vote has not passed. > > The proposed code change has been vetoed by a qualified voter. The > validity of that veto has been called into question since "the voter > must provide with the veto a technical justification showing why the > change is bad (opens a security exposure, negatively affects > performance, etc. )." It has been less than 24 hours since Dongjoon's > veto was called into question. He should be given a chance to explain > why there is technical justification for it. > > On Thu, Mar 13, 2025 at 7:21 PM Jungtaek Lim > <kabhwan.opensou...@gmail.com> wrote: > > > > The vote passes with 7 +1s (3 binding +1s) and 1 -1s (1 binding -1s). > > Thanks to all who helped with the vote! > > > > I'm going to make a code change in branch-4.0 quickly so that we don't > have to trigger another RC for Spark 4.0.0 just because of this. > > > > (* = binding) > > +1: > > - Sean R. Owen * > > - Jungtaek Lim > > - Nicholas Chammas > > - Wenchen Fan * > > - Adam Binford > > - Russell Jurney > > - Yang Jie * > > > > -1: > > - Dongjoon Hyun * > > > > Thanks, > > Jungtaek Lim (HeartSaVioR) > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
