Let me try to break this down another way: I see a few competing concerns, each with QA related time requirements (asserting 8 weeks minimum, 16 weeks maximum we should plan for to stabilize a GA): 1. A freeze to a branch to stabilize for release (8-16 weeks of QA required after we branch) 2. A freeze to a branch to make room for large complex work to have increased velocity on merge due to having a more stable destination (8-16 weeks of QA required after they merge) 3. A commitment to release once a year (for our purposes, we've defined this as calendar year) (8-16 weeks of QA required *before*). If we walk backwards from Dec 1, that means our latest date to freeze and validate a 5.0 branch would be Friday August 11; let's go with 1st Friday in August for simplicity, 2023-08-04. That would give us just over 16 weeks worst-case to stabilize.
So we branch for 5.0 *at the latest* on 2023-08-04; I think we can all agree on this? So the next question: when do we branch for 5.0 *at the earliest*? Pros and cons of an earlier branch: Pros: • Earlier start of validation testing on a more stable base (no improvements or new features excepting CEP-15 and CEP-21) • Theoretically higher velocity of completion of CEP-15 and CEP-21 (the team doing this can speak to the degree to which this is true) Cons: • Smaller amount of improvements and new features go into 5.0 • The rest of the dev community has another branch they need to target with bugfixes (annoying but not _too_ costly since bugfixes are often a bit smaller in scope) Through this lens, we are weighing the belief that CEP-15 and CEP-21 will land by August 1st and be accelerated by branching early against the belief that other improvements and features will go in if we branch later; if we freeze today and neither CEP-15 nor CEP-21 land for unforeseen reasons, we will have a GA release that had a shortened amount of time for new features and improvements to be merged in. Lastly, as input data to the discussion, here's a list of all the new features and improvements in 5.0 as of today; hypothetically were we to freeze 5.0 today and worst-case unforeseen things lead to CEP-21 and CEP-15 not landing by cutoff, this would be our feature-set for our next GA: https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRA%20and%20fixversion%20%3D%205.0%20and%20fixversion%20not%20in%20(4.0.x%2C%204.1.x%2C%204.0%2C%204.1%2C%204.1.1%2C%204.1.2%2C%204.0.8%2C%204.1-alpha1%2C%204.1-alpha1%2C%204.1-beta2%2C%204.1-beta1%2C%204.1-rc1)%20and%20type%20in%20(%22New%20Feature%22%2C%20%22Improvement%22)%20and%20component%20!%3D%20Accord%20order%20by%20type%20desc%2C%20resolved%20desc Phew. Ok, so using the above framework, I'm personally ok with us freezing 5.0 earlier than August 1st if the engineers actively on CEP-15 and CEP-21 indicate that it will appreciably increase their velocity. The list of improvements and features is substantial enough that an earlier freeze would still have enough in it to be "meaty" in my opinion; especially given the likelihood of CEP-25 (Trie-indexed SSTable format) landing relatively soon. So the next question to me is: "when"? On that I defer to Sam, Alex, Benedict, Blake, David, et. al: how much would freezing 5.0 early help in terms of your development velocity on TrM and Accord? On Wed, Apr 19, 2023, at 6:22 AM, Henrik Ingo wrote: > I'm going to repeat the point from my own thread: rather than thinking of > this as some kind of concession to two exceptional CEPs, could we rather take > the point of view that they get their own space and time precisely because > they are large and invasive and both the merge and testing of them will > benefit from everything else in the branch quieting down? > > I'm also not particularly interested in a long feature freeze beyond 1-3 > months that would serve the above purpose well. > > In short: the proposal should not be that everyone else just have to sit > still and wait for two late stragglers. The proposal is merely to organise > work such that we maximise velocity and quality for merging cep-15&21. > Anything beyond that should be judged differently. > > On Tue, 18 Apr 2023, 23:48 J. D. Jordan, <jeremiah.jor...@gmail.com> wrote: >> >> I also don’t really see the value in “freezing with exceptions for two giant >> changes to come after the freeze”. >> >> -Jeremiah >> >>> On Apr 18, 2023, at 1:08 PM, Caleb Rackliffe <calebrackli...@gmail.com> >>> wrote: >>> >>> > Caleb, you appear to be the only one objecting, and it does not appear >>> > that you have made any compromises in this thread. >>> >>> All I'm really objecting to is making special exceptions for particular >>> CEPs in relation to our freeze date. In other words, let's not have a >>> pseudo-freeze date and a "real" freeze date, when the thing that makes the >>> latter supposedly necessary is a very invasive change to the database that >>> risks our desired GA date. Also, again, I don't understand how cutting a >>> 5.0 branch makes anything substantially easier to start testing. Perhaps >>> I'm the only one who thinks this. If so, I'm not going to make further >>> noise about it. >>> >>> On Tue, Apr 18, 2023 at 7:26 AM Henrik Ingo <henrik.i...@datastax.com> >>> wrote: >>>> I forgot one last night: >>>> >>>> From Benjamin we have a question that I think went unanswered? >>>> >>>> *> Should it not facilitate the work if the branch stops changing heavily?* >>>> >>>> This is IMO a good perspective. To me it seems weird to be too hung up on >>>> a "hard limit" on a specific day, when we are talking about merges where a >>>> single merge / rebase takes more than one day. We will have to stop >>>> merging smaller work to trunk anyway, when CEP-21 is being merged. No? >>>> >>>> henrik >>>> >>>> On Tue, Apr 18, 2023 at 3:24 AM Henrik Ingo <henrik.i...@datastax.com> >>>> wrote: >>>>> Trying to collect a few loose ends from across this thread >>>>> >>>>> *> I'm receptive to another definition of "stabilize", * >>>>> >>>>> I think the stabilization period implies more than just CI, which is >>>>> mostly a function of unit tests working correctly. For example, at >>>>> Datastax we have run a "large scale" test with >100 nodes, over several >>>>> weeks, both for 4.0 and 4.1. For obvious reasons such tests can't run in >>>>> nightly CI builds. >>>>> >>>>> Also it is not unusual that during the testing phase developers or >>>>> specialized QA engineers can develop new tests (which are possibly added >>>>> to CI) to improve coverage for and especially targeting new features in >>>>> the release. For example the fixes to Paxos v2 were found by such work >>>>> before 4.1. >>>>> >>>>> Finally, maybe it's a special case relevant only for this release, but >>>>> as a significant part of the Datastax team has been focused on porting >>>>> these large existing features from DSE, and to get them merged before the >>>>> original May date, we also have tens of bug fixes waiting to be >>>>> upstreamed too. (It used to be an even 100, but I'm unsure what the count >>>>> is today.) >>>>> >>>>> In fact! If you are worried about how to occupy yourself between a May >>>>> "soft freeze" and September'ish hard freeze, you are welcome to chug on >>>>> that backlog. The bug fixes are already public and ASL licensed, in the >>>>> 4.0 based branch here >>>>> <https://github.com/datastax/cassandra/commits/ds-trunk>. >>>>> Failed with an unknown error. >>>>> ** >>>>> *> 3a. If we allow merge of CEP-15 / CEP-21 after branch, we risk >>>>> invalidating stabilization and risk our 2023 GA date* >>>>> >>>>> I think this is the assumption that I personally disagree with. If this >>>>> is true, why do we even bother running any CI before the CEP-21 merge? It >>>>> will all be invalidated anyway, right? >>>>> >>>>> In my experience, it is beneficial to test as early as possible, and at >>>>> different checkpoints during development. If we wouldn't do it, and we >>>>> find some issue in late November, then the window to search for the >>>>> commit that introduced the regression is all the way back to the 4.1 GA. >>>>> If on the other hand the same test was already rune during the soft >>>>> freeze, then we can know that we may focus our search onto CEP-15 and >>>>> CEP-21. >>>>> >>>>> >>>>> *> get comfortable with cutting feature previews or snapshot alphas like >>>>> we agreed to for earlier access to new stuff* >>>>> >>>>> Snapshots are in fact a valid compromise proposal: A snapshot would >>>>> provide a constant version / point in time to focus testing on, but on >>>>> the other hand would allow trunk (or the 5.0 branch, in other proposals) >>>>> to remain open to new commits. Somewhat "invalidating" the testing work, >>>>> but presumably the branch will be relatively calm anyway. Which leads me >>>>> to 2 important questions: >>>>> >>>>> *WHO would be actively merging things into 5.0 during June-August? * >>>>> >>>>> By my count at that point I expect most contributors to either furiously >>>>> work on Acccord and TCM, or work on stabilization (tests, fixes). >>>>> >>>>> Also, if someone did contribute new feature code during this time, they >>>>> might find it hard to get priority for reviews, if others are focused on >>>>> the above tasks. >>>>> >>>>> Finally, I expect most Europeans to be on vacation 33% of that time. >>>>> Non-Europeans may want to try it too! >>>>> >>>>> >>>>> *WHAT do we expect to get merged during June-August?* >>>>> >>>>> Compared to the tens of thousands of lines of code being merged by >>>>> Accord, SAI, UCS and Tries... I imagine even the worst case during a >>>>> non-freeze in June-August would be just a tiny percentage of the large >>>>> CEPs. >>>>> >>>>> In this thread I only see Paulo announcing an intent to commit against >>>>> trunk during a soft freeze, and even he agrees with a 5.0 branch freeze. >>>>> >>>>> This last question is basically a form of saying I hope we aren't >>>>> discussing a problem that doesn't even exist? >>>>> >>>>> henrik >>>>> >>>>> -- >>>>> >>>>> >>>>> >>>>> >>>>> *Henrik Ingo* >>>>> >>>>> *c*. +358 40 569 7354 >>>>> >>>>> *w*. _www.datastax.com_ >>>>> >>>>> __ >>>>> <https://urldefense.com/v3/__https://www.facebook.com/datastax__;!!PbtH5S7Ebw!achduQtuq0o7cl-Dk7XvTA8_oLUOJh6l8mpZcv-JslA8QkkQ69uzoh-ye88cvbwPRJraEfynopqui7KfehccGBMIKmYB$> >>>>> __ <https://twitter.com/datastax> __ >>>>> <https://urldefense.com/v3/__https://www.linkedin.com/company/datastax/__;!!PbtH5S7Ebw!achduQtuq0o7cl-Dk7XvTA8_oLUOJh6l8mpZcv-JslA8QkkQ69uzoh-ye88cvbwPRJraEfynopqui7KfehccGAYmsWrL$> >>>>> __ <https://github.com/datastax/> >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> >>>> >>>> >>>> *Henrik Ingo* >>>> >>>> *c*. +358 40 569 7354 >>>> >>>> *w*. _www.datastax.com_ >>>> >>>> __ >>>> <https://urldefense.com/v3/__https://www.facebook.com/datastax__;!!PbtH5S7Ebw!achduQtuq0o7cl-Dk7XvTA8_oLUOJh6l8mpZcv-JslA8QkkQ69uzoh-ye88cvbwPRJraEfynopqui7KfehccGBMIKmYB$> >>>> __ <https://twitter.com/datastax> __ >>>> <https://urldefense.com/v3/__https://www.linkedin.com/company/datastax/__;!!PbtH5S7Ebw!achduQtuq0o7cl-Dk7XvTA8_oLUOJh6l8mpZcv-JslA8QkkQ69uzoh-ye88cvbwPRJraEfynopqui7KfehccGAYmsWrL$> >>>> __ <https://github.com/datastax/> >>>>
