Yes, I'll start a new thread this week to collect the ongoing projects and track the progress. We can discuss there and see if the code freeze date (Feb 1) is reasonable or not. But in any case, we should try our best to stick with the schedule, or we need to vote to change the date.
On Wed, Jan 15, 2025 at 4:29 PM Jungtaek Lim <kabhwan.opensou...@gmail.com> wrote: > Shall we not focus on the date itself but focus on the delivery? > > I don't have any concern about branch cut, but the starting date of the QA > period sounds really tight. We have two weeks remaining, and one week will > be a national holiday of both China and South Korea. Shall we collect and > check the plan of ongoing projects to see whether they can make it in time > in advance, so that we could determine the realistic date of the QA period > rather than an arbitrary one? > > On Wed, Jan 15, 2025 at 5:11 PM Wenchen Fan <cloud0...@gmail.com> wrote: > >> Dongjoon, let's take it easy. I'll cut the branch today to stick with the >> committed schedule (2025-01-15), and I'll pick a time that is >> convenient for me (likely 9 PM UTC+8). As I mentioned earlier, people can >> keep backporting commits to the 4.0 branch before February 1, so we don't >> have any blocker for the branch cut (as long as the CI is green). And I >> don't think the branch cut is an urgent blocker to anything either, so no >> one needs to wait for it today. >> >> For committers, just remember that starting from tomorrow you need to do >> two additional keyboard actions when merging PRs if you want to backport it >> to 4.0. >> >> Thanks, >> Wenchen >> >> On Wed, Jan 15, 2025 at 3:44 PM Dongjoon Hyun <dongjoon.h...@gmail.com> >> wrote: >> >>> Let me enumerate some questions. >>> >>> 1. Since you have no open list, do you have some internal PRs which are >>> planned to be open the rest of Today? >>> >>> 2. For the timezone, the following timezone was originally proposed by >>> Hyukjin and agreed by me. >>> Apparently, you disagree with this because you don't want me to >>> create it at midnight in PST, right? >>> >>> > one alternative would be to cut the branch at midnight in PST >>> > as relatively more commits are coming in daytime in PST. >>> >>> 3. Since it's already 3:44 PM (GMT+8) in China, >>> how many hours do we need to wait until you finish your day? >>> Could you be more specific? >>> >>> > Since it's still within my working hours today, ... after I finish my >>> day. >>> >>> 4. As we know, Apache Spark merge script allows `backporting` with just >>> two additional keyboard actions. >>> Even if I create branch-4.0 now, you can use that feature after >>> having our branch-4.0. >>> It's easy, isn't it? Or, does it still bother you for some reason? >>> Are you going to delete `branch-4.0` created by me? >>> >>> Sincerely, >>> Dongjoon. >>> >>> >>> >>> On Tue, Jan 14, 2025 at 10:55 PM Wenchen Fan <cloud0...@gmail.com> >>> wrote: >>> >>>> I don't have any specific PRs in mind, as I review PRs almost daily: PRs >>>> I've commented on >>>> <https://github.com/apache/spark/pulls?q=commenter%3Acloud-fan+sort%3Acreated-desc> >>>> . >>>> >>>> Since it's still within my working hours today, I’d prefer to do the >>>> branch cut after I finish my day. I hope this is a reasonable request. >>>> >>>> On Wed, Jan 15, 2025 at 2:42 PM Dongjoon Hyun <dongjoon.h...@gmail.com> >>>> wrote: >>>> >>>>> Well, previously, you didn't have specific PRs in your very previous >>>>> email. >>>>> >>>>> > I don't have specific PRs in mind, >>>>> >>>>> Now, it seems that you made up your mind to tell what is going on >>>>> there. Which PRs are those? >>>>> >>>>> > I'm still reviewing and may merge PRs during the daytime. >>>>> >>>>> Let's see what your real concern is and let us help you. >>>>> >>>>> I really want to understand how those PRs are blocked or interrupted >>>>> by `branch cut`. >>>>> >>>>> Dongjoon. >>>>> >>>>> On Tue, Jan 14, 2025 at 10:25 PM Wenchen Fan <cloud0...@gmail.com> >>>>> wrote: >>>>> >>>>>> OK nice joke, I should have said 2025-01-15 23:59:59 PST :) >>>>>> >>>>>> Anyway, I'll cut the branch tonight in my timezone (UTC+8) as I'm >>>>>> still reviewing and may merge PRs during the daytime. The branch cut will >>>>>> still be on 2025-01-15 and please let me know if the policy requires the >>>>>> time to be midnight PST. >>>>>> >>>>>> >>>>>> On Wed, Jan 15, 2025 at 1:57 PM Dongjoon Hyun < >>>>>> dongjoon.h...@gmail.com> wrote: >>>>>> >>>>>>> We provide sufficient information in advance on our website and >>>>>>> mailing list. >>>>>>> >>>>>>> Exactly one week ago, we also had the dev mailing threads about >>>>>>> `Spark 4.0.0 timeline`. >>>>>>> >>>>>>> https://lists.apache.org/thread/vq00tmwz6x4ddjypk2rmw8ktotty0pb6 >>>>>>> (2025-01-07) >>>>>>> >>>>>>> In addition, Apache Spark also complains about the proposed date, >>>>>>> `2025-01-15 24:00:00 PST`, is invalid. >>>>>>> >>>>>>> ``` >>>>>>> spark-sql (default)> SELECT CAST('2025-01-15 23:59:59' AS TIMESTAMP); >>>>>>> 2025-01-15 23:59:59 >>>>>>> Time taken: 0.045 seconds, Fetched 1 row(s) >>>>>>> spark-sql (default)> SELECT CAST('2025-01-15 24:00:00' AS TIMESTAMP); >>>>>>> [CAST_INVALID_INPUT] The value '2025-01-15 24:00:00' of the >>>>>>> type "STRING" cannot be cast to "TIMESTAMP" because it is malformed. >>>>>>> ``` >>>>>>> >>>>>>> BTW, after creating the `branch-4.0`, the following PR will be merged >>>>>>> in order to differentiate `master` from `branch-4.0`. >>>>>>> >>>>>>> [SPARK-50822][BUILD] Setting version to 4.1.0-SNAPSHOT >>>>>>> https://github.com/apache/spark/pull/49495 >>>>>>> >>>>>>> Best regards, >>>>>>> Dongjoon. >>>>>>> >>>>>>> >>>>>>> On Tue, Jan 14, 2025 at 9:03 PM Wenchen Fan <cloud0...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> I don't have specific PRs in mind, I'm just referring to the huge >>>>>>>> amount of commits we merge every day: >>>>>>>> https://github.com/apache/spark/commits/master >>>>>>>> >>>>>>>> BTW, 4 hours seems too short for a reminder, why not 2025-01-15 >>>>>>>> 24:00:00 PST? I can do it as it will be my afternoon at my timezone. >>>>>>>> >>>>>>>> On Wed, Jan 15, 2025 at 11:56 AM Hyukjin Kwon <gurwls...@apache.org> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I don't have any examples :-). Just wanted to make sure. +1 for >>>>>>>>> cutting soon as scheduled. >>>>>>>>> >>>>>>>>> On Wed, 15 Jan 2025 at 12:55, Dongjoon Hyun < >>>>>>>>> dongjoon.h...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Thank you, Wenchen and Hyukjin. >>>>>>>>>> >>>>>>>>>> BTW, if you can, please provide specific PRs which you are >>>>>>>>>> referring to as active. >>>>>>>>>> >>>>>>>>>> I can help those PRs as much as possible if I can. >>>>>>>>>> >>>>>>>>>> For `branch-4.0`, as Hyukjin mentioned, I'm going to create it at >>>>>>>>>> midnight PST (Today, 2025-01-15 00:00:00 PST). >>>>>>>>>> >>>>>>>>>> In other words, we have about 4 hours from now. >>>>>>>>>> >>>>>>>>>> Dongjoon. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Jan 14, 2025 at 7:15 PM Hyukjin Kwon < >>>>>>>>>> gurwls...@apache.org> wrote: >>>>>>>>>> >>>>>>>>>>> one alternative would be to cut the branch at midnight in PST as >>>>>>>>>>> relatively more commits are coming in daytime in PST. >>>>>>>>>>> >>>>>>>>>>> On Wed, 15 Jan 2025 at 12:12, Hyukjin Kwon <gurwls...@apache.org> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Yeah, I agree that cutting in the middle of week, e.g., commits >>>>>>>>>>>> are being merged, it might cause some confusions which branch >>>>>>>>>>>> committers >>>>>>>>>>>> should target for a while. >>>>>>>>>>>> >>>>>>>>>>>> If we're all happy with that, I am fine going ahead as >>>>>>>>>>>> scheduled. >>>>>>>>>>>> >>>>>>>>>>>> On Wed, 15 Jan 2025 at 12:03, Wenchen Fan <cloud0...@gmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I don’t have a strong preference, as people can backport >>>>>>>>>>>>> whatever they need to the 4.0 branch before February 1. My >>>>>>>>>>>>> intention was to >>>>>>>>>>>>> avoid interrupting others’ work in the middle of a week when PRs >>>>>>>>>>>>> are >>>>>>>>>>>>> actively being merged, and I thought the weekend would be a >>>>>>>>>>>>> better time. >>>>>>>>>>>>> However, I’m fine with either way. >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Jan 15, 2025 at 10:40 AM Dongjoon Hyun < >>>>>>>>>>>>> dongjoon.h...@gmail.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi, Wenchen. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'd not delay the branch cut. We had better stick to our plan >>>>>>>>>>>>>> and those week days to stablize branch-4.0. >>>>>>>>>>>>>> >>>>>>>>>>>>>> If you are busy, I'm going to cut the branch on schedule >>>>>>>>>>>>>> (2015-01-15) and help the on-going PR reviews. >>>>>>>>>>>>>> >>>>>>>>>>>>>> As you mentioned, we don't block feature backportings. There >>>>>>>>>>>>>> is no reason to break the community schedule. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Sincerely, >>>>>>>>>>>>>> Dongjoon >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Jan 14, 2025 at 17:52 Wenchen Fan < >>>>>>>>>>>>>> cloud0...@gmail.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> According to the schedule >>>>>>>>>>>>>>> <https://spark.apache.org/versioning-policy.html>, the 4.0 >>>>>>>>>>>>>>> branch cut is set for January 15. I plan to cut the branch at >>>>>>>>>>>>>>> the end of >>>>>>>>>>>>>>> this week so that folks can have a few more working days to >>>>>>>>>>>>>>> complete the >>>>>>>>>>>>>>> ongoing work. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Note: While we can still merge PRs to the 4.0 branch after >>>>>>>>>>>>>>> the cut, code freeze will begin on February 1. After that date, >>>>>>>>>>>>>>> only bug >>>>>>>>>>>>>>> fixes can be backported. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Wenchen >>>>>>>>>>>>>>> >>>>>>>>>>>>>>
