Isn't it a bit excessive to talk about "making the voice of the community very clear" when only a very small fraction of Spark users have participated in the discussion or cast their votes?
That said, I read the initial emails and understood the proposal. Since it seemed obvious and straightforward to me, I didn’t pay much attention to the ongoing discussion. However, after this long thread, I’m now unclear about the issue—why is there hesitation in dropping a configuration that was never supposed to be there in the first place, especially in a major release like Spark 4? Rather than going through the entire email chain, I’d appreciate a quick confirmation: I support keeping it as is in Spark 3.5 and removing it in Spark 4. Would that be considered a -1 or a +1? (non-binding.) Thanks, El dom, 16 mar 2025, 3:36, Jungtaek Lim <kabhwan.opensou...@gmail.com> escribió: > Thanks Mark for trying to make the voice from community very clear and be > recorded. Now community is required to just validate Dongjoon’s veto based > on his last post and there is no longer a concern about vote process with > your proposal. > If we agree that Dongjoon’s justification is moot, I’ll need to update the > result to NOT contain Dongjoon’s -1, as it is not justified, and it’s > passed with no veto even if we consider the prior VOTE to be code change. > > I agree this is lot clearer way for us to push community to express their > voice very seriously, including their justification super clearly, and > stand on “one side” rather than trying to nitpick the process to block it. > Thanks again Mark, this is awesome. > > 2025년 3월 16일 (일) 오전 8:36, Mark Hamstra <markhams...@gmail.com>님이 작성: > >> I don’t think we quite need to involve the board yet. There is still the >> open question of whether the technical justification for the veto is valid, >> and it is purely up to the PMC to resolve those doubts. >> >> I don’t understand the apparent reluctance to call for resolution of that >> question even though the technical justification (and thus the veto) has >> been challenged in comments, so I’ll do it myself momentarily…. >> >> >> On Sat, Mar 15, 2025 at 2:33 PM Holden Karau <holden.ka...@gmail.com> >> wrote: >> >>> My $0.02: >>> >>> I do not believe that this vote has passed. I believe there is a valid >>> veto. On a personal level from a migration point of view I think Spark 4 is >>> the perfect time to drop this configuration. >>> >>> Given the disagreement of if this is a valid veto I think we should >>> pause until the board has been given time to provide its input. >>> >>> Previous perceived dismissing of committer and PMC input has resulted in >>> this being a more sensitive topic in this project than perhaps some others. >>> >>> How do folks feel about writing up their views for the board sending it >>> over to them and seeing if they want to provide input? >>> >>> Twitter: https://twitter.com/holdenkarau >>> Fight Health Insurance: https://www.fighthealthinsurance.com/ >>> <https://www.fighthealthinsurance.com/?q=hk_email> >>> Books (Learning Spark, High Performance Spark, etc.): >>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>> Pronouns: she/her >>> >>> >>> On Sat, Mar 15, 2025 at 1:38 PM Mich Talebzadeh < >>> mich.talebza...@gmail.com> wrote: >>> >>>> This is my gist >>>> >>>> Mark from your passionate language I gather you see this as a "Code >>>> Change" veto. Your reasoning seems to be straightforward, i.e. the vote's >>>> purpose is to decide whether to add code (migration logic) to the Spark 4.0 >>>> branch. In your view, the outcome of the vote directly alters the >>>> software's code? >>>> >>>> However, If we see it as a *procedural matter only* like some others >>>> include myself, it involves a vote and the interpretation of rules. >>>> >>>> In summary >>>> >>>> 1. If it is a code change vote, a -1 can be seen as a veto, >>>> blocking the change unless specific conditions are met (like the PMC >>>> overriding the veto). >>>> 2. If it is just a procedural vote, a -1 might simply be a >>>> dissenting vote, *not necessarily carrying the power to block the >>>> entire action.* >>>> >>>> FYI, I recall I voted -1 (non binding) on another thread and >>>> Dongioon asked me to explain which it was in his right >>>> >>>> I can see the following vote cast (1) >>>> >>>> - Jungtaek Lim: +1 (non-binding) >>>> - Sean Owen: +1 to retain >>>> - Yang Jie: -1, later withdraws it and casts +1 >>>> - Adam Binford: +1 (non-binding) >>>> - Russell Jurney: +1 non-binding >>>> - Yang Jie: +1 >>>> - Mridul Muralidharan: +1 >>>> - Dongjoon Hyun: -1 >>>> >>>> >>>> *(1) Summary of Voting from 21 emails in the attached file* from >>>> https://lists.apache.org/thread/nm3p1zjcybdl0p0mc56t2rl92hb9837n : >>>> >>>> For Retaining Migration Logic (+1): Jungtaek Lim, Sean Owen, Yang Jie >>>> (initially -1, then +1), Adam Binford (non-binding), Russell Jurney >>>> (non-binding), Mridul Muralidharan >>>> Against Retaining Migration Logic (-1): Dongjoon Hyun >>>> >>>> Maybe we should put a bar on it and allow Dongjoon to qualify his >>>> statement as 1 or 2 above, thern it could be escalated if needed or put at >>>> rest >>>> >>>> HTH >>>> >>>> >>>> >>>> >>>> >>>> On Sat, 15 Mar 2025 at 18:26, Mark Hamstra <markhams...@gmail.com> >>>> wrote: >>>> >>>>> That is utter nonsense, Sean! You do not have any authority to declare >>>>> the matter concluded, and I will escalate to the board if you persist >>>>> in this approach. >>>>> >>>>> The proposed code change has been vetoed. As I delineated previously, >>>>> there are two and only two ways forward under the ASF Voting Process. >>>>> That does not include any individual simply declaring that the matter >>>>> has been concluded regardless of the veto and ASF process. >>>>> >>>>> On Sat, Mar 15, 2025 at 11:18 AM Sean Owen <sro...@gmail.com> wrote: >>>>> > >>>>> > Mark et al - this thread has gone on way too long. Everyone has >>>>> expressed their opinion. The result stands. >>>>> > Anyone who is really upset about it, please escalate to the board or >>>>> something, but, this thread and decision point has now concluded. >>>>> > >>>>> > >>>>> > On Sat, Mar 15, 2025 at 1:16 PM Mark Hamstra <markhams...@gmail.com> >>>>> wrote: >>>>> >> >>>>> >> You do not have the authority to declare Dongjoon's technical >>>>> >> justification invalid. That is up to the PMC: "In case of doubt, >>>>> >> deciding whether a technical justification is valid is up to the >>>>> PMC." >>>>> >> >>>>> >> On Sat, Mar 15, 2025 at 6:20 AM Jungtaek Lim >>>>> >> <kabhwan.opensou...@gmail.com> wrote: >>>>> >> > >>>>> >> > To summarize, the main arguments of both proposals are "whether >>>>> we can force users to upgrade to Spark 3.5.5 first before upgrading Spark >>>>> 4.0.0" vs "we should include migration logic to Spark 4.0.0 because that >>>>> is >>>>> not realistic". Where is the "technical objection" here? If you say there >>>>> was politics I can clearly say never, but even if you interpret there was >>>>> politics, politics is not "technical objection". I can quote the relevant >>>>> ASF page for you. >>>>> >> > >>>>> >> > https://www.apache.org/foundation/voting.html#Veto >>>>> >> > >>>>> >> > > A -1 vote by a qualified voter stops a code-modification >>>>> proposal in its tracks. This constitutes a veto, and it cannot be >>>>> overruled >>>>> nor overridden by anyone. Vetoes stand until and unless the individual >>>>> withdraws their veto. >>>>> >> > > To prevent vetoes from being used capriciously, the voter must >>>>> provide with the veto a technical justification showing why the change is >>>>> bad (opens a security exposure, negatively affects performance, etc. ). A >>>>> veto without a justification is invalid and has no weight. >>>>> >> > >>>>> >> > The justification must be "technical one" for vote. I hope ASF >>>>> just lists the most cases rather than leaving this as etc, but I think ASF >>>>> believes individual's judgement, and I claim there is no "technical >>>>> reason". Having to put 4 more lines is never a technical reason. It is >>>>> never meant to be used for blocking different opinions. It must be used >>>>> for >>>>> blocking "incidents which impact users", while we are here to do the >>>>> opposite, saving users' life. >>>>> >> > >>>>> >> > On Sat, Mar 15, 2025 at 9:51 PM Jungtaek Lim < >>>>> kabhwan.opensou...@gmail.com> wrote: >>>>> >> >> >>>>> >> >> > That's the reason why you proposed the vote procedure and we >>>>> agreed. >>>>> >> >> >>>>> >> >> Didn’t you see the part “we agreed”? Who is we in the context? >>>>> >> >> >>>>> >> >> I don’t think he answered my questions - he explained his >>>>> reasoning of his proposal which majorly does not agree with. You even said >>>>> uou are not persuaded and I want to ask you now you were persuaded from >>>>> his >>>>> last post. >>>>> >> >> >>>>> >> >> Again I haven’t heard my answers. He showed his reasoning but >>>>> there is nothing about the evidence of the validity of “technical” >>>>> objection. I think I have asked people who judged his -1 as veto for their >>>>> reasoning of how this could be “technical” objection and I don’t think I >>>>> heard anything. >>>>> >> >> >>>>> >> >> I can be corrected if you can point out what is the “technical” >>>>> objection. If you or Dongjoon do not provide this to the end of the week, >>>>> I >>>>> have to consider I haven’t heard about that and the veto (although >>>>> Dongjoon >>>>> stated it is not a veto) will be ignored. >>>>> >> >> >>>>> >> >> 2025년 3월 15일 (토) 오후 8:19, Mark Hamstra <markhams...@gmail.com>님이 >>>>> 작성: >>>>> >> >>> >>>>> >> >>> Once again, I have to object. Dongjoon said that the vote is a >>>>> time >>>>> >> >>> limited procedure, not that the vote itself is a procedural >>>>> vote as >>>>> >> >>> distinct from a code change vote or a package release vote. >>>>> >> >>> >>>>> >> >>> Frankly, this feels like you are trying to manipulate the vote >>>>> >> >>> procedure by misrepresenting Dongjoon, and you are quickly >>>>> losing my >>>>> >> >>> confidence in your ability to administer a fair voting >>>>> procedure. >>>>> >> >>> >>>>> >> >>> I still consider the proposal to be vetoed. >>>>> >> >>> >>>>> >> >>> >>>>> >> >>> On Fri, Mar 14, 2025 at 6:11 PM Jungtaek Lim >>>>> >> >>> <kabhwan.opensou...@gmail.com> wrote: >>>>> >> >>> > >>>>> >> >>> > UPDATE: >>>>> >> >>> > >>>>> >> >>> > We were having a discussion about the type of VOTE, since >>>>> Dongjoon's -1 should be considered as a veto if we see this as a code >>>>> change VOTE. >>>>> >> >>> > Dongjoon clarified that he does not see this VOTE as a code >>>>> change, hence he gave -1 but not intended to block the VOTE. >>>>> >> >>> > >>>>> >> >>> > That said, we have confirmed that Dongjoon's -1 is not a >>>>> veto. I think the VOTE result is correct as it is. I'll proceed with the >>>>> next steps. >>>>> >> >>> > >>>>> >> >>> > On Fri, Mar 14, 2025 at 11:19 AM Jungtaek Lim < >>>>> kabhwan.opensou...@gmail.com> wrote: >>>>> >> >>> >> >>>>> >> >>> >> The vote passes with 7 +1s (3 binding +1s) and 1 -1s (1 >>>>> binding -1s). >>>>> >> >>> >> Thanks to all who helped with the vote! >>>>> >> >>> >> >>>>> >> >>> >> I'm going to make a code change in branch-4.0 quickly so >>>>> that we don't have to trigger another RC for Spark 4.0.0 just because of >>>>> this. >>>>> >> >>> >> >>>>> >> >>> >> (* = binding) >>>>> >> >>> >> +1: >>>>> >> >>> >> - Sean R. Owen * >>>>> >> >>> >> - Jungtaek Lim >>>>> >> >>> >> - Nicholas Chammas >>>>> >> >>> >> - Wenchen Fan * >>>>> >> >>> >> - Adam Binford >>>>> >> >>> >> - Russell Jurney >>>>> >> >>> >> - Yang Jie * >>>>> >> >>> >> >>>>> >> >>> >> -1: >>>>> >> >>> >> - Dongjoon Hyun * >>>>> >> >>> >> >>>>> >> >>> >> Thanks, >>>>> >> >>> >> Jungtaek Lim (HeartSaVioR) >>>>> >> >>> >>>>> >> >>> >>>>> --------------------------------------------------------------------- >>>>> >> >>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>>>> >> >>> >>>>> >> >>>>> >> >>>>> --------------------------------------------------------------------- >>>>> >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>>>> >> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>>>> >>>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>> >>>
