I would like to conclude this vote with my own -1, for the same reasons as
Holden.
Personally, I feel that this approach has been overly excessive in several
ways - such as overriding a veto - and it would have been better to explore
alternative solutions first.
That said, it seems that this vote has passed by a simple majority, and the
vote in the other thread remains valid.
Since it has now been 48 hours, as mentioned earlier in the thread, I hope
we can formally conclude this vote.
On Tue, 18 Mar 2025 at 06:47, Yuanjian Li <xyliyuanj...@gmail.com> wrote:
> +1
>
> I would like to add my perspective to this discussion. It's important that
> we maintain a fair and objective technical evaluation process, especially
> when a veto is cast. A veto should be backed by concrete technical
> reasoning.
>
> From my reading of the discussion, the key concern is whether the
> justification for the veto is based on an objective technical issue with
> the proposed change. If the concern is not rooted in a clear technical
> limitation or regression, then it should be open to further discussion and
> reconsideration. It is crucial for us as a community to ensure that
> decisions are made transparently and with technical merit as the primary
> driver.
>
> I also want to emphasize that maintaining a welcoming and inclusive
> community is vital to the success of Apache Spark. If contributors feel
> isolated or that decisions are being swayed by affiliations rather than
> technical arguments, that would set a concerning precedent. We should
> strive to engage in discussions constructively and focus on what is best
> for the project and its users.
>
> Jungtaek Lim <kabhwan.opensou...@gmail.com> 于2025年3月16日周日 23:11写道：
>
>> The problem you are mentioning is arguably resolvable if we leave a
>> config name as it is, and add
>> withAlternative("spark.databricks.sql.optimizer.pruneFiltersCanPruneStreamingSubplan").
>> Let's not nitpick just from reverting the PR. We have to revert the PR
>> "semantically".
>>
>> Btw, from what I understand of dealing with backport PR is, we mostly
>> merge in master to begin with, and down the version line. When we handle my
>> migration PR, we do not follow this practice without any discussion. I
>> submitted PRs for master/4.0/3.5, and 3.5 was merged "first", and when I
>> asked for merging to 4.0, I was pushed back. I don't think this is an
>> ordinary practice of dealing with multiple versions of PRs, especially
>> since the author never agreed with the way of processing.
>>
>> If we were following the practice, we should already have migration logic
>> for master/4.0/3.5, since we should have the same fix in master/4.0. We
>> probably had a discussion about removing config from master/4.0 based on
>> the discussion, and we probably agreed to remove the config since we still
>> have a migration logic. W.r.t migration logic, based on the discussion we
>> are having, we probably can't make an agreement to take it out, then
>> arguably the migration logic is left as it is.
>>
>> This way I never needed to drive such a long and sensitive DISCUSSION and
>> VOTE. But the PR for master and 4.0 weren't merged because of the
>> individual's belief of the rollout plan, which we are seeing does not fit a
>> majority of voices now.
>>
>> Shall we fix the broken process in which we made a huge mistake before
>> moving on? We should have merged the same content in 3.5 to master/4.0 as
>> well, and then have a PR to remove the config. This is totally swapped
>> which does not make sense to me.
>>
>> On Mon, Mar 17, 2025 at 1:57 PM Dongjoon Hyun <dongj...@apache.org>
>> wrote:
>>
>>> I reviewed Wenchen's reverting PR. Although it's a proposal for
>>> discussion, it is another breaking change against Apache Spark 3.5.5, isn't
>>> it? If we consider Apache Spark 3.5.4 users, I believe we need to consider
>>> Apache Spark 3.5.5 users too which uses
>>> `spark.sql.optimizer.pruneFiltersCanPruneStreamingSubplan` already.
>>>
>>> https://github.com/apache/spark/pull/50291
>>>
>>> - buildConf("spark.sql.optimizer.pruneFiltersCanPruneStreamingSubplan")
>>> +
>>> buildConf("spark.databricks.sql.optimizer.pruneFiltersCanPruneStreamingSubplan")
>>>
>>> Dongjoon.
>>>
>>> On 2025/03/17 02:36:42 Wenchen Fan wrote:
>>> > I've created the revert PR for branch-4.0:
>>> > https://github.com/apache/spark/pull/50291 . We can merge PRs with
>>> lazy
>>> > consensus but it's clear that this breaking change PR has failed to
>>> achieve
>>> > consensus.
>>> >
>>> > I hope we now have a clear foundation for discussing solutions. As it
>>> > stands, the misnamed configuration will be released in 4.0.0. I like
>>> > Jungtaek’s proposal to deprecate it, but the decision is up to the
>>> > community.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>
>>>
