Re: [DISCUSS] CASSANDRA-13994
> > Also, what is the plan for cutting beta branch? I am still learning the > Apache/C* way so excuse me if I missed something. Looking at the Lifecycle > document, I see a point only about GA major version branch. > My understanding is that we are already in freeze for big changes. When > would be the right time for creating a beta branch? Something more than > closing the last beta blockers tickets? My understanding from the doc is that creating the release branch `cassandra-4.0` happens with GA. That means we're working on trunk until GA. Previously C* hasn't had any branches but these release branches, i.e. no beta or rc branches… (afaik) I would have thought it better to branch on RC? if at that point much of the testing efforts are subsiding, and folk are ready for other things.
Re: [DISCUSS] CASSANDRA-13994
The new release lifecycle guarantees changes these dynamics. Given our commitment to API stability between beta 1 and GA, this has introduced a time where changes being deferred to the next major due to API modification will be unable to be merged unless we branch upon beta. This warrants another discuss thread I think rather than nesting it here. I'll take that on. On Fri, Jun 26, 2020 at 4:38 AM Mick Semb Wever wrote: > > > > Also, what is the plan for cutting beta branch? I am still learning the > > Apache/C* way so excuse me if I missed something. Looking at the > Lifecycle > > document, I see a point only about GA major version branch. > > My understanding is that we are already in freeze for big changes. When > > would be the right time for creating a beta branch? Something more than > > closing the last beta blockers tickets? > > > > My understanding from the doc is that creating the release branch > `cassandra-4.0` happens with GA. That means we're working on trunk until > GA. Previously C* hasn't had any branches but these release branches, i.e. > no beta or rc branches… (afaik) > > I would have thought it better to branch on RC? if at that point much of > the testing efforts are subsiding, and folk are ready for other things. >
[DISCUSS] When to branch 4.0
As we close on cutting beta1, a new consequence of our release lifecycle is becoming apparent. With guarantees of API stability in the beta phase, any work that is deferred from alpha to the next major due to API impacting changes will atrophy for as long as the beta is active. Cutting a branch for the 4.0 line upon release of beta1 would mitigate this problem and allow work in flight to be merged in, as well as unblock the work of others who may not be focusing on testing 4.0, whether it be due to their interest and focus or due to saturation on the work in scope for the release. The obvious downsides to cutting a branch of a major and allowing dev on trunk to continue separately is 1) the need to merge up to trunk on changes going into beta, and 2) a risk of a lack of focus on testing the release due to the availability of developing on trunk. My personal thoughts on those two points: 1) changes going into beta should be small enough that a fast-forward merge should be available in the majority of cases up to trunk. We've done this with previous releases and it wasn't prohibitively expensive in terms of time. Further, I would posit that changes going into beta should be on the smaller side, further mitigating the burden of this process. 2) We've been feature frozen since late 2018 with the expressed intention to encourage work on testing and stabilizing 4.0. I am not aware of any contributors whose time and energy has been invested in testing 4.0 that would otherwise have gone to trunk (i.e. this approach achieving its desired outcomes), however I do know of several major contributors and contributions that have atrophied and been deterred from further work on the project due to waiting for 4.0 to release. I don't think it's appropriate for us as an existing body of contributors to mandate either how each other or more detrimentally how other new contributors engage with and contribute to the project by disallowing contributions past 4.0; I take the position that we should do everything reasonably possible to encourage a diversity of contributions to the project. I deeply believe that making deliberate decisions to prioritize new engagement and interaction on the project as the context in which it's used evolves is vital to the project's health long term. That's just my .02 - I'd be curious to hear what everyone else thinks. ~Josh
Re: [DISCUSS] When to branch 4.0
Fwiw, I agree with that POV in general (that it's probably beneficial on balance to branch now/soonish for the reasonings Josh mentioned). -- Sylvain On Fri, Jun 26, 2020 at 3:43 PM Joshua McKenzie wrote: > As we close on cutting beta1, a new consequence of our release lifecycle is > becoming apparent. With guarantees of API stability in the beta phase, any > work that is deferred from alpha to the next major due to API impacting > changes will atrophy for as long as the beta is active. > > Cutting a branch for the 4.0 line upon release of beta1 would mitigate this > problem and allow work in flight to be merged in, as well as unblock the > work of others who may not be focusing on testing 4.0, whether it be due to > their interest and focus or due to saturation on the work in scope for the > release. > > The obvious downsides to cutting a branch of a major and allowing dev on > trunk to continue separately is 1) the need to merge up to trunk on changes > going into beta, and 2) a risk of a lack of focus on testing the release > due to the availability of developing on trunk. My personal thoughts on > those two points: > > 1) changes going into beta should be small enough that a fast-forward merge > should be available in the majority of cases up to trunk. We've done this > with previous releases and it wasn't prohibitively expensive in terms of > time. Further, I would posit that changes going into beta should be on the > smaller side, further mitigating the burden of this process. > > 2) We've been feature frozen since late 2018 with the expressed intention > to encourage work on testing and stabilizing 4.0. I am not aware of any > contributors whose time and energy has been invested in testing 4.0 that > would otherwise have gone to trunk (i.e. this approach achieving its > desired outcomes), however I do know of several major contributors and > contributions that have atrophied and been deterred from further work on > the project due to waiting for 4.0 to release. > > I don't think it's appropriate for us as an existing body of contributors > to mandate either how each other or more detrimentally how other new > contributors engage with and contribute to the project by disallowing > contributions past 4.0; I take the position that we should do everything > reasonably possible to encourage a diversity of contributions to the > project. I deeply believe that making deliberate decisions to prioritize > new engagement and interaction on the project as the context in which it's > used evolves is vital to the project's health long term. > > That's just my .02 - I'd be curious to hear what everyone else thinks. > > ~Josh >
Re: [DISCUSS] When to branch 4.0
I think that this goes well with the philosophy of the open source and volunteering contributions. Also, it could really open the door for more new features and people contributing to the project in the long term. Ekaterina On Fri, 26 Jun 2020 at 10:44, Sylvain Lebresne wrote: > Fwiw, I agree with that POV in general (that it's probably beneficial on > balance to branch now/soonish for the reasonings Josh mentioned). > -- > Sylvain > > > On Fri, Jun 26, 2020 at 3:43 PM Joshua McKenzie > wrote: > > > As we close on cutting beta1, a new consequence of our release lifecycle > is > > becoming apparent. With guarantees of API stability in the beta phase, > any > > work that is deferred from alpha to the next major due to API impacting > > changes will atrophy for as long as the beta is active. > > > > Cutting a branch for the 4.0 line upon release of beta1 would mitigate > this > > problem and allow work in flight to be merged in, as well as unblock the > > work of others who may not be focusing on testing 4.0, whether it be due > to > > their interest and focus or due to saturation on the work in scope for > the > > release. > > > > The obvious downsides to cutting a branch of a major and allowing dev on > > trunk to continue separately is 1) the need to merge up to trunk on > changes > > going into beta, and 2) a risk of a lack of focus on testing the release > > due to the availability of developing on trunk. My personal thoughts on > > those two points: > > > > 1) changes going into beta should be small enough that a fast-forward > merge > > should be available in the majority of cases up to trunk. We've done this > > with previous releases and it wasn't prohibitively expensive in terms of > > time. Further, I would posit that changes going into beta should be on > the > > smaller side, further mitigating the burden of this process. > > > > 2) We've been feature frozen since late 2018 with the expressed intention > > to encourage work on testing and stabilizing 4.0. I am not aware of any > > contributors whose time and energy has been invested in testing 4.0 that > > would otherwise have gone to trunk (i.e. this approach achieving its > > desired outcomes), however I do know of several major contributors and > > contributions that have atrophied and been deterred from further work on > > the project due to waiting for 4.0 to release. > > > > I don't think it's appropriate for us as an existing body of contributors > > to mandate either how each other or more detrimentally how other new > > contributors engage with and contribute to the project by disallowing > > contributions past 4.0; I take the position that we should do everything > > reasonably possible to encourage a diversity of contributions to the > > project. I deeply believe that making deliberate decisions to prioritize > > new engagement and interaction on the project as the context in which > it's > > used evolves is vital to the project's health long term. > > > > That's just my .02 - I'd be curious to hear what everyone else thinks. > > > > ~Josh > > >
Re: [DISCUSS] When to branch 4.0
Thanks for bringing this up Josh. I think the last time we all discussed this on the mailing list was during the initial freeze thread where we agreed "that between the September freeze date and beta, a new branch would not be created and trunk would only have bug fixes and performance improvements committed to it." Now that we are closer to beta and have a more formal governance model I think its good to revisit. I'm torn. Folks should absolutely be able to scratch an itch as part of the ethos of the project but we also haven't made substantial progress on the testing epic -- less than I expected when I was +1 to branch at beta in the initial proposal. A few general thoughts come to mind around this: - Does not having a target branch truly discourage folks from scratching an itch? Is the lack of a timeline on when they could actually see that merge in a release more substantial? - Regarding timeline (and scope), I wonder if we would be better branching after we have a bit more of an idea of our future process and development / release lifecycle. Perhaps we should discuss that first? - I haven't seen any CEPs, etc for major features. These discussions aren't blocked by the freeze and would presumably precede any need to merge to trunk? - For smaller changes that don't need CEPs, I know maintaining a long running branch can be a pain but I would like to better understand how many of these are truly out there before asking the committers to maintain and merge into a 4th branch (which is not super challenging but is measurable overhead). Jordan On Fri, Jun 26, 2020 at 6:43 AM Joshua McKenzie wrote: > As we close on cutting beta1, a new consequence of our release lifecycle is > becoming apparent. With guarantees of API stability in the beta phase, any > work that is deferred from alpha to the next major due to API impacting > changes will atrophy for as long as the beta is active. > > Cutting a branch for the 4.0 line upon release of beta1 would mitigate this > problem and allow work in flight to be merged in, as well as unblock the > work of others who may not be focusing on testing 4.0, whether it be due to > their interest and focus or due to saturation on the work in scope for the > release. > > The obvious downsides to cutting a branch of a major and allowing dev on > trunk to continue separately is 1) the need to merge up to trunk on changes > going into beta, and 2) a risk of a lack of focus on testing the release > due to the availability of developing on trunk. My personal thoughts on > those two points: > > 1) changes going into beta should be small enough that a fast-forward merge > should be available in the majority of cases up to trunk. We've done this > with previous releases and it wasn't prohibitively expensive in terms of > time. Further, I would posit that changes going into beta should be on the > smaller side, further mitigating the burden of this process. > > 2) We've been feature frozen since late 2018 with the expressed intention > to encourage work on testing and stabilizing 4.0. I am not aware of any > contributors whose time and energy has been invested in testing 4.0 that > would otherwise have gone to trunk (i.e. this approach achieving its > desired outcomes), however I do know of several major contributors and > contributions that have atrophied and been deterred from further work on > the project due to waiting for 4.0 to release. > > I don't think it's appropriate for us as an existing body of contributors > to mandate either how each other or more detrimentally how other new > contributors engage with and contribute to the project by disallowing > contributions past 4.0; I take the position that we should do everything > reasonably possible to encourage a diversity of contributions to the > project. I deeply believe that making deliberate decisions to prioritize > new engagement and interaction on the project as the context in which it's > used evolves is vital to the project's health long term. > > That's just my .02 - I'd be curious to hear what everyone else thinks. > > ~Josh >
Re: [DISCUSS] When to branch 4.0
I was originally against the trunk freeze because I didn't want it to prevent progress, now I'm 100% on board with it and I think we should keep it in place till we release 4.0.0. Branching anytime before we 4.0.0 adds extra burden to the folks trying to get 4.0.0 out the door (because of merge up), which means it'll take longer. Anyone taking issue with how long it's taken so far to get 4.0 out should recognize that if we branch and start allowing new features, we're going to take even longer to get 4.0 out. At this point 4.0 is turning into somewhat of a running joke, like Duke Nukem Forever. We can't have the next release come faster, merge new features, and have quality be up to the bar we've all aimed to achieve (classic good fast cheap scenario). We've got a tradeoff here. If folks are advocating we branch anytime before we release & unfreeze trunk, they'd better be prepared for the consequences of an even further delayed release. Did anyone ever think we'd possibly be frozen till 2021? After 4.0, we *really* need to start focusing on refactoring the codebase to improve modularity and testability so we don't have to ever go through this multi-year process again, because it's incredibly unhealthy to have to freeze trunk this long. I think a lot of people are frustrated, and I get it, but I don't think the path to improving our collective position is to say "screw it" and branch any earlier than the 4.0.0 release. On Fri, Jun 26, 2020 at 9:26 AM Jordan West wrote: > Thanks for bringing this up Josh. I think the last time we all discussed > this on the mailing list was during the initial freeze thread where we > agreed "that between the September freeze date and beta, a new branch would > not be created and trunk would only have bug fixes and performance > improvements committed to it." Now that we are closer to beta and have a > more formal governance model I think its good to revisit. > > I'm torn. Folks should absolutely be able to scratch an itch as part of the > ethos of the project but we also haven't made substantial progress on the > testing epic -- less than I expected when I was +1 to branch at beta in the > initial proposal. A few general thoughts come to mind around this: > > - Does not having a target branch truly discourage folks from scratching an > itch? Is the lack of a timeline on when they could actually see that merge > in a release more substantial? > > - Regarding timeline (and scope), I wonder if we would be better branching > after we have a bit more of an idea of our future process and development / > release lifecycle. Perhaps we should discuss that first? > > - I haven't seen any CEPs, etc for major features. These discussions aren't > blocked by the freeze and would presumably precede any need to merge to > trunk? > > - For smaller changes that don't need CEPs, I know maintaining a long > running branch can be a pain but I would like to better understand how many > of these are truly out there before asking the committers to maintain and > merge into a 4th branch (which is not super challenging but is measurable > overhead). > > Jordan > > On Fri, Jun 26, 2020 at 6:43 AM Joshua McKenzie > wrote: > > > As we close on cutting beta1, a new consequence of our release lifecycle > is > > becoming apparent. With guarantees of API stability in the beta phase, > any > > work that is deferred from alpha to the next major due to API impacting > > changes will atrophy for as long as the beta is active. > > > > Cutting a branch for the 4.0 line upon release of beta1 would mitigate > this > > problem and allow work in flight to be merged in, as well as unblock the > > work of others who may not be focusing on testing 4.0, whether it be due > to > > their interest and focus or due to saturation on the work in scope for > the > > release. > > > > The obvious downsides to cutting a branch of a major and allowing dev on > > trunk to continue separately is 1) the need to merge up to trunk on > changes > > going into beta, and 2) a risk of a lack of focus on testing the release > > due to the availability of developing on trunk. My personal thoughts on > > those two points: > > > > 1) changes going into beta should be small enough that a fast-forward > merge > > should be available in the majority of cases up to trunk. We've done this > > with previous releases and it wasn't prohibitively expensive in terms of > > time. Further, I would posit that changes going into beta should be on > the > > smaller side, further mitigating the burden of this process. > > > > 2) We've been feature frozen since late 2018 with the expressed intention > > to encourage work on testing and stabilizing 4.0. I am not aware of any > > contributors whose time and energy has been invested in testing 4.0 that > > would otherwise have gone to trunk (i.e. this approach achieving its > > desired outcomes), however I do know of several major contributors and > > contributions that have atrophied and been dete
Re: [DISCUSS] When to branch 4.0
I was speaking with Ekaterina and Benjamin about some of the currently alpha scoped work which is what made me think of this dynamic. There's a very different dynamic from "if we push this out to the next major, this code will atrophy for 3-6 months" vs. "if we push this out to the next major, you should still be able to merge this shortly so it's not as high a merge cost." Given our difficulty getting 4.0 out the door, my original thinking was that having *less* disincentives to pushing out optional work would help us make more objective decisions about when to hold a release for a ticket vs. when to push the ticket. I suspect we'll go through something similar when looking at the scope during beta for non-API breaking code that could instead live in a patch release or a follow-up major. In my opinion we lack clarity around scoping of this release and don't seem to have a project-wide consensus on how we're handling this. I'm reading an assumption that this is about CEP's and major features; while there's a non-trivial body of work on the wiki as draft CEP's at this time, those have not been brought to the mailing list out of respect for multiple direct requests earlier this year not to distract from work on getting 4.0 out the door. Branching anytime before we 4.0.0 adds extra burden to the folks trying to > get 4.0.0 out the door (because of merge up) Given both that we've done this with every major release in the past, as well as the type of work we'd expect to land during the beta phase (smaller, non-api breaking, defect fixing or smaller performance tuning), I didn't personally originally weigh this as being as much of a burden as you perceive it to be. I'm curious to hear more about this. ~Josh On Fri, Jun 26, 2020 at 1:12 PM Jon Haddad wrote: > I was originally against the trunk freeze because I didn't want it to > prevent progress, now I'm 100% on board with it and I think we should keep > it in place till we release 4.0.0. > > Branching anytime before we 4.0.0 adds extra burden to the folks trying to > get 4.0.0 out the door (because of merge up), which means it'll take > longer. Anyone taking issue with how long it's taken so far to get 4.0 out > should recognize that if we branch and start allowing new features, we're > going to take even longer to get 4.0 out. At this point 4.0 is turning > into somewhat of a running joke, like Duke Nukem Forever. > > We can't have the next release come faster, merge new features, and have > quality be up to the bar we've all aimed to achieve (classic good fast > cheap scenario). We've got a tradeoff here. If folks are advocating we > branch anytime before we release & unfreeze trunk, they'd better be > prepared for the consequences of an even further delayed release. Did > anyone ever think we'd possibly be frozen till 2021? > > After 4.0, we *really* need to start focusing on refactoring the codebase > to improve modularity and testability so we don't have to ever go through > this multi-year process again, because it's incredibly unhealthy to have to > freeze trunk this long. I think a lot of people are frustrated, and I get > it, but I don't think the path to improving our collective position is to > say "screw it" and branch any earlier than the 4.0.0 release. > > > > On Fri, Jun 26, 2020 at 9:26 AM Jordan West wrote: > > > Thanks for bringing this up Josh. I think the last time we all discussed > > this on the mailing list was during the initial freeze thread where we > > agreed "that between the September freeze date and beta, a new branch > would > > not be created and trunk would only have bug fixes and performance > > improvements committed to it." Now that we are closer to beta and have a > > more formal governance model I think its good to revisit. > > > > I'm torn. Folks should absolutely be able to scratch an itch as part of > the > > ethos of the project but we also haven't made substantial progress on the > > testing epic -- less than I expected when I was +1 to branch at beta in > the > > initial proposal. A few general thoughts come to mind around this: > > > > - Does not having a target branch truly discourage folks from scratching > an > > itch? Is the lack of a timeline on when they could actually see that > merge > > in a release more substantial? > > > > - Regarding timeline (and scope), I wonder if we would be better > branching > > after we have a bit more of an idea of our future process and > development / > > release lifecycle. Perhaps we should discuss that first? > > > > - I haven't seen any CEPs, etc for major features. These discussions > aren't > > blocked by the freeze and would presumably precede any need to merge to > > trunk? > > > > - For smaller changes that don't need CEPs, I know maintaining a long > > running branch can be a pain but I would like to better understand how > many > > of these are truly out there before asking the committers to maintain and > > merge into a 4th branch (which is not super c
Re: [DISCUSS] When to branch 4.0
--> Branching anytime before we 4.0.0 adds extra burden to the folks trying to get 4.0.0 This. This is all that matters IMO. Some have been saying 4.0.0 is very delayed, and that this harms the project - and they're right. So I'm surprised to see those same voices advocating we prioritise their feature development instead? On 26/06/2020, 18:12, "Jon Haddad" wrote: I was originally against the trunk freeze because I didn't want it to prevent progress, now I'm 100% on board with it and I think we should keep it in place till we release 4.0.0. Branching anytime before we 4.0.0 adds extra burden to the folks trying to get 4.0.0 out the door (because of merge up), which means it'll take longer. Anyone taking issue with how long it's taken so far to get 4.0 out should recognize that if we branch and start allowing new features, we're going to take even longer to get 4.0 out. At this point 4.0 is turning into somewhat of a running joke, like Duke Nukem Forever. We can't have the next release come faster, merge new features, and have quality be up to the bar we've all aimed to achieve (classic good fast cheap scenario). We've got a tradeoff here. If folks are advocating we branch anytime before we release & unfreeze trunk, they'd better be prepared for the consequences of an even further delayed release. Did anyone ever think we'd possibly be frozen till 2021? After 4.0, we *really* need to start focusing on refactoring the codebase to improve modularity and testability so we don't have to ever go through this multi-year process again, because it's incredibly unhealthy to have to freeze trunk this long. I think a lot of people are frustrated, and I get it, but I don't think the path to improving our collective position is to say "screw it" and branch any earlier than the 4.0.0 release. On Fri, Jun 26, 2020 at 9:26 AM Jordan West wrote: > Thanks for bringing this up Josh. I think the last time we all discussed > this on the mailing list was during the initial freeze thread where we > agreed "that between the September freeze date and beta, a new branch would > not be created and trunk would only have bug fixes and performance > improvements committed to it." Now that we are closer to beta and have a > more formal governance model I think its good to revisit. > > I'm torn. Folks should absolutely be able to scratch an itch as part of the > ethos of the project but we also haven't made substantial progress on the > testing epic -- less than I expected when I was +1 to branch at beta in the > initial proposal. A few general thoughts come to mind around this: > > - Does not having a target branch truly discourage folks from scratching an > itch? Is the lack of a timeline on when they could actually see that merge > in a release more substantial? > > - Regarding timeline (and scope), I wonder if we would be better branching > after we have a bit more of an idea of our future process and development / > release lifecycle. Perhaps we should discuss that first? > > - I haven't seen any CEPs, etc for major features. These discussions aren't > blocked by the freeze and would presumably precede any need to merge to > trunk? > > - For smaller changes that don't need CEPs, I know maintaining a long > running branch can be a pain but I would like to better understand how many > of these are truly out there before asking the committers to maintain and > merge into a 4th branch (which is not super challenging but is measurable > overhead). > > Jordan > > On Fri, Jun 26, 2020 at 6:43 AM Joshua McKenzie > wrote: > > > As we close on cutting beta1, a new consequence of our release lifecycle > is > > becoming apparent. With guarantees of API stability in the beta phase, > any > > work that is deferred from alpha to the next major due to API impacting > > changes will atrophy for as long as the beta is active. > > > > Cutting a branch for the 4.0 line upon release of beta1 would mitigate > this > > problem and allow work in flight to be merged in, as well as unblock the > > work of others who may not be focusing on testing 4.0, whether it be due > to > > their interest and focus or due to saturation on the work in scope for > the > > release. > > > > The obvious downsides to cutting a branch of a major and allowing dev on > > trunk to continue separately is 1) the need to merge up to trunk on > changes > > going into beta, and 2) a risk of a lack of focus on testing the release > > due to the availability of developing on trunk. My personal thoughts on > > those two points: > > > > 1) changes going into beta should be small enough that a fast-forward > merge > > should be av
Re: [DISCUSS] When to branch 4.0
Ok, I'm sorry for reaching the opposite conclusion before reading this email - the implication here instead appears to be that, you believe, people are advocating to integrate work that should be deferred - is that correct? Perhaps we should have a direct conversation about the tickets you consider this to be the case for? FWIW, I don't think this is a rational strategy for appeasing anybody pushing for inclusion in this release, given that cutting a new branch says nothing about the timeline for its release. On 26/06/2020, 19:58, "Joshua McKenzie" wrote: I was speaking with Ekaterina and Benjamin about some of the currently alpha scoped work which is what made me think of this dynamic. There's a very different dynamic from "if we push this out to the next major, this code will atrophy for 3-6 months" vs. "if we push this out to the next major, you should still be able to merge this shortly so it's not as high a merge cost." Given our difficulty getting 4.0 out the door, my original thinking was that having *less* disincentives to pushing out optional work would help us make more objective decisions about when to hold a release for a ticket vs. when to push the ticket. I suspect we'll go through something similar when looking at the scope during beta for non-API breaking code that could instead live in a patch release or a follow-up major. In my opinion we lack clarity around scoping of this release and don't seem to have a project-wide consensus on how we're handling this. I'm reading an assumption that this is about CEP's and major features; while there's a non-trivial body of work on the wiki as draft CEP's at this time, those have not been brought to the mailing list out of respect for multiple direct requests earlier this year not to distract from work on getting 4.0 out the door. Branching anytime before we 4.0.0 adds extra burden to the folks trying to > get 4.0.0 out the door (because of merge up) Given both that we've done this with every major release in the past, as well as the type of work we'd expect to land during the beta phase (smaller, non-api breaking, defect fixing or smaller performance tuning), I didn't personally originally weigh this as being as much of a burden as you perceive it to be. I'm curious to hear more about this. ~Josh On Fri, Jun 26, 2020 at 1:12 PM Jon Haddad wrote: > I was originally against the trunk freeze because I didn't want it to > prevent progress, now I'm 100% on board with it and I think we should keep > it in place till we release 4.0.0. > > Branching anytime before we 4.0.0 adds extra burden to the folks trying to > get 4.0.0 out the door (because of merge up), which means it'll take > longer. Anyone taking issue with how long it's taken so far to get 4.0 out > should recognize that if we branch and start allowing new features, we're > going to take even longer to get 4.0 out. At this point 4.0 is turning > into somewhat of a running joke, like Duke Nukem Forever. > > We can't have the next release come faster, merge new features, and have > quality be up to the bar we've all aimed to achieve (classic good fast > cheap scenario). We've got a tradeoff here. If folks are advocating we > branch anytime before we release & unfreeze trunk, they'd better be > prepared for the consequences of an even further delayed release. Did > anyone ever think we'd possibly be frozen till 2021? > > After 4.0, we *really* need to start focusing on refactoring the codebase > to improve modularity and testability so we don't have to ever go through > this multi-year process again, because it's incredibly unhealthy to have to > freeze trunk this long. I think a lot of people are frustrated, and I get > it, but I don't think the path to improving our collective position is to > say "screw it" and branch any earlier than the 4.0.0 release. > > > > On Fri, Jun 26, 2020 at 9:26 AM Jordan West wrote: > > > Thanks for bringing this up Josh. I think the last time we all discussed > > this on the mailing list was during the initial freeze thread where we > > agreed "that between the September freeze date and beta, a new branch > would > > not be created and trunk would only have bug fixes and performance > > improvements committed to it." Now that we are closer to beta and have a > > more formal governance model I think its good to revisit. > > > > I'm torn. Folks should absolutely be able to scratch an itch as part of > the > > ethos of the project but we also haven't made substantial progress on the > > testing epic -- less than I expected when I was +1 to branch at beta in > the > > initial proposal. A few general thoughts come to mind around this: > > > > - Does not h
Re: [DISCUSS] When to branch 4.0
> > > Branching anytime before we 4.0.0 adds extra burden to the folks trying > to > > get 4.0.0 out the door (because of merge up) > > Given both that we've done this with every major release in the past, as > well as the type of work we'd expect to land during the beta phase > (smaller, non-api breaking, defect fixing or smaller performance tuning), I > didn't personally originally weigh this as being as much of a burden as you > perceive it to be. Looking at this a different way, you might say we have previously cut the release branch somewhere around beta. Because previous releases haven't (all) had so much focus on testing and alphas. My impression is that 4.0.0 will be closer compared to a second or third patch of previous major releases. So I would have thought it makes sense around beta or RC to branch, especially RC because between RC and GA it is more about a cooling period, public acceptance and testing. That RC to GA window should be quiet enough that it makes sense to branch then, and kick off the CEP discussions. I don't think the forward merging really is so much of a problem, it's a normal activity in the C* codebase, and the additional merge-forward window between either beta or RC, to GA is short. Thanks Ekaterina and Benjamin and Josh for raising the discussion.
Re: [DISCUSS] When to branch 4.0
We currently have 2.1, 2.2, 3.0 3.11, and trunk. With a new branch we'll _also_ have whatever is next, let's call it 5.0. Nothing is stopping us for discussing CEPs now, and nothing prevents folks from making their own feature branches. If we're going to add another branch (4.0) and let people merge to trunk, we're making an *active* decision to push the 4.0 release out even further, because the folks working on it will have to learn the new code when they merge forward. I'm -1 on branching before we release 4.0. On Fri, Jun 26, 2020 at 2:04 PM Mick Semb Wever wrote: > > > > > Branching anytime before we 4.0.0 adds extra burden to the folks trying > > to > > > get 4.0.0 out the door (because of merge up) > > > > Given both that we've done this with every major release in the past, as > > well as the type of work we'd expect to land during the beta phase > > (smaller, non-api breaking, defect fixing or smaller performance > tuning), I > > didn't personally originally weigh this as being as much of a burden as > you > > perceive it to be. > > > > Looking at this a different way, you might say we have previously cut the > release branch somewhere around beta. Because previous releases haven't > (all) had so much focus on testing and alphas. My impression is that 4.0.0 > will be closer compared to a second or third patch of previous major > releases. > > So I would have thought it makes sense around beta or RC to branch, > especially RC because between RC and GA it is more about a cooling period, > public acceptance and testing. That RC to GA window should be quiet enough > that it makes sense to branch then, and kick off the CEP discussions. > > I don't think the forward merging really is so much of a problem, it's a > normal activity in the C* codebase, and the additional merge-forward window > between either beta or RC, to GA is short. > > Thanks Ekaterina and Benjamin and Josh for raising the discussion. >
Re: [DISCUSS] When to branch 4.0
> Nothing is stopping us for discussing CEPs now, and nothing prevents folks > from making their own feature branches. I disagree. CEPs are just as - if not more - of a distraction than branching. Work doesn't happen in a vacuum. Those who are today focusing what resources they can on shipping 4.0.0 will have to divert resources to the new CEP and feature development happening on the project. It is unrealistic to expect otherwise. We can have a swifter 4.0.0 release, or we can begin earnestly developing new features, but we cannot have both. On 26/06/2020, 22:09, "Jon Haddad" wrote: We currently have 2.1, 2.2, 3.0 3.11, and trunk. With a new branch we'll _also_ have whatever is next, let's call it 5.0. Nothing is stopping us for discussing CEPs now, and nothing prevents folks from making their own feature branches. If we're going to add another branch (4.0) and let people merge to trunk, we're making an *active* decision to push the 4.0 release out even further, because the folks working on it will have to learn the new code when they merge forward. I'm -1 on branching before we release 4.0. On Fri, Jun 26, 2020 at 2:04 PM Mick Semb Wever wrote: > > > > > Branching anytime before we 4.0.0 adds extra burden to the folks trying > > to > > > get 4.0.0 out the door (because of merge up) > > > > Given both that we've done this with every major release in the past, as > > well as the type of work we'd expect to land during the beta phase > > (smaller, non-api breaking, defect fixing or smaller performance > tuning), I > > didn't personally originally weigh this as being as much of a burden as > you > > perceive it to be. > > > > Looking at this a different way, you might say we have previously cut the > release branch somewhere around beta. Because previous releases haven't > (all) had so much focus on testing and alphas. My impression is that 4.0.0 > will be closer compared to a second or third patch of previous major > releases. > > So I would have thought it makes sense around beta or RC to branch, > especially RC because between RC and GA it is more about a cooling period, > public acceptance and testing. That RC to GA window should be quiet enough > that it makes sense to branch then, and kick off the CEP discussions. > > I don't think the forward merging really is so much of a problem, it's a > normal activity in the C* codebase, and the additional merge-forward window > between either beta or RC, to GA is short. > > Thanks Ekaterina and Benjamin and Josh for raising the discussion. > - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: [DISCUSS] When to branch 4.0
On Fri, Jun 26, 2020 at 2:58 PM Benedict Elliott Smith wrote: > > Nothing is stopping us for discussing CEPs now, and nothing prevents > folks from making their own feature branches. > > I disagree. CEPs are just as - if not more - of a distraction than > branching. > > Work doesn't happen in a vacuum. Those who are today focusing what > resources they can on shipping 4.0.0 will have to divert resources to the > new CEP and feature development happening on the project. It is > unrealistic to expect otherwise. > > We can have a swifter 4.0.0 release, or we can begin earnestly developing > new features, but we cannot have both. > > Agreed 100% and I would prefer to see us all focus on getting 4.0.0 out. I only meant we never explicitly voted to prevent CEPs from being submitted at this time and it was more in response to a comment in the initial email in this thread. > > On 26/06/2020, 22:09, "Jon Haddad" wrote: > > We currently have 2.1, 2.2, 3.0 3.11, and trunk. With a new branch > we'll > _also_ have whatever is next, let's call it 5.0. > > Nothing is stopping us for discussing CEPs now, and nothing prevents > folks > from making their own feature branches. > > If we're going to add another branch (4.0) and let people merge to > trunk, > we're making an *active* decision to push the 4.0 release out even > further, > because the folks working on it will have to learn the new code when > they > merge forward. > > I'm -1 on branching before we release 4.0. > > On Fri, Jun 26, 2020 at 2:04 PM Mick Semb Wever > wrote: > > > > > > > > Branching anytime before we 4.0.0 adds extra burden to the folks > trying > > > to > > > > get 4.0.0 out the door (because of merge up) > > > > > > Given both that we've done this with every major release in the > past, as > > > well as the type of work we'd expect to land during the beta phase > > > (smaller, non-api breaking, defect fixing or smaller performance > > tuning), I > > > didn't personally originally weigh this as being as much of a > burden as > > you > > > perceive it to be. > > > > > > > > Looking at this a different way, you might say we have previously > cut the > > release branch somewhere around beta. Because previous releases > haven't > > (all) had so much focus on testing and alphas. My impression is that > 4.0.0 > > will be closer compared to a second or third patch of previous major > > releases. > > > > So I would have thought it makes sense around beta or RC to branch, > > especially RC because between RC and GA it is more about a cooling > period, > > public acceptance and testing. That RC to GA window should be quiet > enough > > that it makes sense to branch then, and kick off the CEP discussions. > > > > I don't think the forward merging really is so much of a problem, > it's a > > normal activity in the C* codebase, and the additional merge-forward > window > > between either beta or RC, to GA is short. > > > > Thanks Ekaterina and Benjamin and Josh for raising the discussion. > > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
Re: [DISCUSS] When to branch 4.0
> > 1) changes going into beta should be small enough that a fast-forward merge > should be available in the majority of cases up to trunk. We've done this > with previous releases and it wasn't prohibitively expensive in terms of > time. Further, I would posit that changes going into beta should be on the > smaller side, further mitigating the burden of this process. I don't have much experience here, but what I find is that a decent chunk of the fixes I have been posting have to be rewritten for 2.2 (mostly CI), 3.0, 3.11, and trunk. This is currently already painful and makes me less likely to want to fix things... If we branch 4.0 and people start refactoring trunk, then there is even more burden to fix issues; I don't see this as a small issue today. 2) We've been feature frozen since late 2018 with the expressed intention > to encourage work on testing and stabilizing 4.0. I am not aware of any > contributors whose time and energy has been invested in testing 4.0 that > would otherwise have gone to trunk (i.e. this approach achieving its > desired outcomes), however I do know of several major contributors and > contributions that have atrophied and been deterred from further work on > the project due to waiting for 4.0 to release. When I joined this project I heard that Repair was a pinpoint for many, so the first thing I did was look into why and started trying to fix bugs with Repair. In doing this I found that the testing of repair is lacking and that small regressions would not be caught by the current tests, so shifted my focus to improving the testing rather than make the large changes I wanted to make; my reasoning was simple, "if I can not rely on the existing tests to show I didn't break anything, by fixing 1 problem I may break 10 others". I was then later pulled off this into 3.x issues as many still struggle upgrading from 2.1 to 3.x. I am currently weighted down by features which are broken in 3.x (so have to rewrite them to be able to migrate), and a constant stream of new data corruption bugs (I currently have 5 on my plate). As I resolve the issues I see the common trend is "we lacked testing in X". Given the amount of data loss and corruption bugs that keep popping up, I do feel it's reasonable to ask people to focus on testing rather than new features. If we don't have a strong foundation we believe in, how do we build a strong house on top? I take the position that we should do everything > reasonably possible to encourage a diversity of contributions to the > project. I deeply believe that making deliberate decisions to prioritize > new engagement and interaction on the project as the context in which it's > used evolves is vital to the project's health long term. I fully agree with this, which is why I feel the feature freeze is in the best interest of the project and that we should be focusing on testing. The hardest thing I have found is that adding a new feature is that there is an unreasonable expectation of what you have to learn, their interactions, and the ability to test their impact. Even simple things become hard given the fact only committers can run Jenkins tests, or you pay money to be able to run the tests... If the intent is to make it easier for new people to contribute to the project, shouldn't the focus be on fixing their pain points such as testing? On Fri, Jun 26, 2020 at 3:38 PM Jordan West wrote: > On Fri, Jun 26, 2020 at 2:58 PM Benedict Elliott Smith < > bened...@apache.org> > wrote: > > > > Nothing is stopping us for discussing CEPs now, and nothing prevents > > folks from making their own feature branches. > > > > I disagree. CEPs are just as - if not more - of a distraction than > > branching. > > > > > Work doesn't happen in a vacuum. Those who are today focusing what > > resources they can on shipping 4.0.0 will have to divert resources to the > > new CEP and feature development happening on the project. It is > > unrealistic to expect otherwise. > > > > We can have a swifter 4.0.0 release, or we can begin earnestly developing > > new features, but we cannot have both. > > > > > Agreed 100% and I would prefer to see us all focus on getting 4.0.0 out. I > only meant we never explicitly voted to prevent CEPs from being submitted > at this time and it was more in response to a comment in the initial email > in this thread. > > > > > > On 26/06/2020, 22:09, "Jon Haddad" wrote: > > > > We currently have 2.1, 2.2, 3.0 3.11, and trunk. With a new branch > > we'll > > _also_ have whatever is next, let's call it 5.0. > > > > Nothing is stopping us for discussing CEPs now, and nothing prevents > > folks > > from making their own feature branches. > > > > If we're going to add another branch (4.0) and let people merge to > > trunk, > > we're making an *active* decision to push the 4.0 release out even > > further, > > because the folks working on it will have to learn the new code when > > they > > m
Re: [DISCUSS] When to branch 4.0
This is all really key to be honest. The only feature we need to develop today is quality, and this is true regardless of 4.0.0. I've seen a lot of talk from some quarters of a new approach to quality, but so far there have been few contributions from the same quarters that materially contribute to it. With a shift to discussing new feature development, it begins to look a bit empty. Put some of that excitement for developing new features into developing new approaches to assuring quality. I would love to see some CEP on this topic, and I would consider it a worthwhile distraction to participate in those discussions. On 26/06/2020, 23:45, "David Capwell" wrote: > > 1) changes going into beta should be small enough that a fast-forward merge > should be available in the majority of cases up to trunk. We've done this > with previous releases and it wasn't prohibitively expensive in terms of > time. Further, I would posit that changes going into beta should be on the > smaller side, further mitigating the burden of this process. I don't have much experience here, but what I find is that a decent chunk of the fixes I have been posting have to be rewritten for 2.2 (mostly CI), 3.0, 3.11, and trunk. This is currently already painful and makes me less likely to want to fix things... If we branch 4.0 and people start refactoring trunk, then there is even more burden to fix issues; I don't see this as a small issue today. 2) We've been feature frozen since late 2018 with the expressed intention > to encourage work on testing and stabilizing 4.0. I am not aware of any > contributors whose time and energy has been invested in testing 4.0 that > would otherwise have gone to trunk (i.e. this approach achieving its > desired outcomes), however I do know of several major contributors and > contributions that have atrophied and been deterred from further work on > the project due to waiting for 4.0 to release. When I joined this project I heard that Repair was a pinpoint for many, so the first thing I did was look into why and started trying to fix bugs with Repair. In doing this I found that the testing of repair is lacking and that small regressions would not be caught by the current tests, so shifted my focus to improving the testing rather than make the large changes I wanted to make; my reasoning was simple, "if I can not rely on the existing tests to show I didn't break anything, by fixing 1 problem I may break 10 others". I was then later pulled off this into 3.x issues as many still struggle upgrading from 2.1 to 3.x. I am currently weighted down by features which are broken in 3.x (so have to rewrite them to be able to migrate), and a constant stream of new data corruption bugs (I currently have 5 on my plate). As I resolve the issues I see the common trend is "we lacked testing in X". Given the amount of data loss and corruption bugs that keep popping up, I do feel it's reasonable to ask people to focus on testing rather than new features. If we don't have a strong foundation we believe in, how do we build a strong house on top? I take the position that we should do everything > reasonably possible to encourage a diversity of contributions to the > project. I deeply believe that making deliberate decisions to prioritize > new engagement and interaction on the project as the context in which it's > used evolves is vital to the project's health long term. I fully agree with this, which is why I feel the feature freeze is in the best interest of the project and that we should be focusing on testing. The hardest thing I have found is that adding a new feature is that there is an unreasonable expectation of what you have to learn, their interactions, and the ability to test their impact. Even simple things become hard given the fact only committers can run Jenkins tests, or you pay money to be able to run the tests... If the intent is to make it easier for new people to contribute to the project, shouldn't the focus be on fixing their pain points such as testing? On Fri, Jun 26, 2020 at 3:38 PM Jordan West wrote: > On Fri, Jun 26, 2020 at 2:58 PM Benedict Elliott Smith < > bened...@apache.org> > wrote: > > > > Nothing is stopping us for discussing CEPs now, and nothing prevents > > folks from making their own feature branches. > > > > I disagree. CEPs are just as - if not more - of a distraction than > > branching. > > > > > Work doesn't happen in a vacuum. Those who are today focusing what > > resources they can on shipping 4.0.0 will have to divert resources to the > > new CEP and feature development happening on the project. It is > > unrealistic to expect otherwise. > > > > We can have a sw
Re: [DISCUSS] When to branch 4.0
> On Jun 26, 2020, at 3:45 PM, David Capwell wrote: > > the ability to test their impact. Even simple things become hard given the > fact only committers can run Jenkins tests, or you pay money to be able to > run the tests... If the intent is to make it easier for new people to > contribute to the project, shouldn't the focus be on fixing their pain > points such as testing? +1 on not branching and keeping focus on testing and fixing 4.0. I am sorry about the situation for non-committers. I tried reaching out to legal and infra in the past without a great response. If someone in the community has a way to reach out and get clarity on problems affecting our contributors, it would be great. Otherwise, I will try to bug them again. Dinesh - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: [DISCUSS] When to branch 4.0
> > I've seen a lot of talk from some quarters of a new approach to quality, > but so far there have been few contributions from the same quarters > Just a heads up - this comes across as passive aggressive sniping. I'm sure you didn't mean it as such but it does read that way (at least to me). When it comes to quality, much like you said in another thread Benedict I think we all need to be honest with ourselves. There's been a lot of talk from *all* quarters but outside a lot of expression of intent across many fronts (verbal, ML, JIRA, slack), very little has publically materialized on the project to this point that I know of. I cleared out assignees on 40_quality_testing tickets earlier this week (overloading shepherds in this field was a mistake IMO - that's on me) which has clarified for some contributors that they can take that work on. There's still considerable uncertainty as to what the scope is for those tickets and nobody really replied to Jordan pinging shepherds for clarification a long while ago. On Fri, Jun 26, 2020 at 8:44 PM Dinesh Joshi wrote: > > On Jun 26, 2020, at 3:45 PM, David Capwell wrote: > > > > the ability to test their impact. Even simple things become hard given > the > > fact only committers can run Jenkins tests, or you pay money to be able > to > > run the tests... If the intent is to make it easier for new people to > > contribute to the project, shouldn't the focus be on fixing their pain > > points such as testing? > > +1 on not branching and keeping focus on testing and fixing 4.0. > > I am sorry about the situation for non-committers. I tried reaching out to > legal and infra in the past without a great response. If someone in the > community has a way to reach out and get clarity on problems affecting our > contributors, it would be great. Otherwise, I will try to bug them again. > > Dinesh > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
Re: [DISCUSS] When to branch 4.0
Great point re: increasing the public visibility of testing and validation activity. I’ll partner with a few contributors I work with to prep a summary of our findings that aren’t already captured in JIRA tickets we’ve filed against 3.x and 4.0 so far (as David mentioned, many issues we’re identifying are 3.0 regressions). Some of these findings are notable, and many are quite standard — the type of experience that’s built from prep to deploy a major piece of software at large scale with a high degree of automation and confidence. But the pieces that are “standard” are the category of work that anyone automating large Cassandra 4.0 deployments must undertake and are things Cassandra’s users will need to be aware of. I’m sure those findings will be valuable to all on this list. Not all blocking issues involve a combination of range tombstones, legacylayout, complex types, paging state, mixed-version clusters, and reverse iteration. :) — Scott > On Jun 26, 2020, at 7:10 PM, Joshua McKenzie wrote: > > >> >> >> I've seen a lot of talk from some quarters of a new approach to quality, >> but so far there have been few contributions from the same quarters >> > Just a heads up - this comes across as passive aggressive sniping. I'm sure > you didn't mean it as such but it does read that way (at least to me). > > When it comes to quality, much like you said in another thread Benedict I > think we all need to be honest with ourselves. There's been a lot of talk > from *all* quarters but outside a lot of expression of intent across many > fronts (verbal, ML, JIRA, slack), very little has publically materialized > on the project to this point that I know of. > > I cleared out assignees on 40_quality_testing tickets earlier this week > (overloading shepherds in this field was a mistake IMO - that's on me) > which has clarified for some contributors that they can take that work on. > There's still considerable uncertainty as to what the scope is for those > tickets and nobody really replied to Jordan pinging shepherds for > clarification a long while ago. > > > > On Fri, Jun 26, 2020 at 8:44 PM Dinesh Joshi wrote: > On Jun 26, 2020, at 3:45 PM, David Capwell wrote: >>> >>> the ability to test their impact. Even simple things become hard given >> the >>> fact only committers can run Jenkins tests, or you pay money to be able >> to >>> run the tests... If the intent is to make it easier for new people to >>> contribute to the project, shouldn't the focus be on fixing their pain >>> points such as testing? >> >> +1 on not branching and keeping focus on testing and fixing 4.0. >> >> I am sorry about the situation for non-committers. I tried reaching out to >> legal and infra in the past without a great response. If someone in the >> community has a way to reach out and get clarity on problems affecting our >> contributors, it would be great. Otherwise, I will try to bug them again. >> >> Dinesh >> - >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org >> For additional commands, e-mail: dev-h...@cassandra.apache.org >> >> - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
