After some further discussions with folks offline, I'd like to revive this discussion.
As Kurt mentioned, to keep it simple I if we can simply build consensus around what is in for 4.0 and what is out. We can then start the process of working off a 4.0 branch towards betas and release candidates. Again as Kurt mentioned, assigning a timeline to it right now is difficult, but having a firm line in the sand around what features/patches are in, then limiting future 4.0 work to bug fixes will give folks a less nebulous target to work on. The other thing to mention is that once we have a 4.0 branch to work off, we at Instaclustr have a commitment to dogfooding the release candidates on our internal staging and internal production workloads before 4.0 becomes generally available. I know other folks have similar commitments and simply having a 4.0 branch with a clear list of things that are in or out will allow everyone to start testing and driving towards a quality release. The other thing is that there are already a large number of changes ready for 4.0, I would suggest not recommending tickets for 4.0 that have not yet been finished/have outstanding work unless you are the person working on it (or are offering to work on it instead) and can get it ready for review in a timely fashion. That way we can build a more realistic working target. For major breaking changes, there is always 5.0 or 4.1 or whatever we end up doing :) Cheers Ben On Thu, Feb 15, 2018 at 9:39 PM kurt greaves <k...@instaclustr.com> wrote: > I don't believe Q3/Q4 is realistic, but I may be biased (or jaded). It's >> possible Q3/Q4 alpha/beta is realistic, but definitely not a release. > > Well, this mostly depends on how much stuff to include in 4.0. Either way > it's not terribly important. If people think 2019 is more realistic we can > aim for that. As I said, it's just a rough timeframe to keep in mind. > > 3.10 was released in January 2017, and we've got around 180 changes for > 4.0 so far, and let's be honest, 3.11 is still pretty young so it's going > to be a significant effort to properly test and verify 4.0. > Let's just stick to getting a list of changes for the moment. I probably > shouldn't have mentioned timeframes, let's just keep in mind that we > shouldn't have such a large set of changes for 4.0 that it takes us years > to complete. > > All that said, what I really care about is building confidence in the >> release, which means an extended testing cycle. If all of those patches >> landed tomorrow, I'd still expect us to be months away from a release, >> because we need to bake the next major - there's too many changes to throw >> out an alpha/beta/rc and hope someone actually runs it. > > Yep. As I said, I'll follow up about testing after we sort out what we're > actually going to include in 4.0. No point trying to come up with a testing > plan for > > On 13 February 2018 at 04:25, Jeff Jirsa <jji...@gmail.com> wrote: > >> >> Advantages of cutting a release sooner than later: >> 1) The project needs to constantly progress forward. Releases are the >> most visible part of that. >> 2) Having a huge changelog in a release increases the likelihood of bugs >> that take time to find. >> >> Advantages of a slower release: >> 1) We don't do major versions often, and when we do breaking changes >> (protocol, file format, etc), we should squeeze in as many as possible to >> avoid having to roll new majors >> 2) There are probably few people actually running 3.11 at scale, so >> probably few people actually testing trunk. >> >> In terms of "big" changes I'd like to see land, the ones that come to >> mind are: >> >> https://issues.apache.org/jira/browse/CASSANDRA-9754 - "Birch" (changes >> file format) >> https://issues.apache.org/jira/browse/CASSANDRA-13442 - Transient >> Replicas (probably adds new replication strategy or similar) >> https://issues.apache.org/jira/browse/CASSANDRA-13628 - Rest of the >> internode netty stuff (no idea if this changes internode stuff, but I bet >> it's a lot easier if it lands on a major) >> https://issues.apache.org/jira/browse/CASSANDRA-7622 - Virtual Tables >> (selfish inclusion, probably doesn't need to be a major at all, and I >> wouldn't even lose sleep if it slips, but I'd like to see it land) >> >> Stuff I'm ok with slipping to 4.X or 5.0, but probably needs to land on a >> major because we'll change something big (like gossip, or the way schema is >> passed, etc): >> >> https://issues.apache.org/jira/browse/CASSANDRA-9667 - Strongly >> consistent membership >> https://issues.apache.org/jira/browse/CASSANDRA-10699 - Strongly >> consistent schema >> >> All that said, what I really care about is building confidence in the >> release, which means an extended testing cycle. If all of those patches >> landed tomorrow, I'd still expect us to be months away from a release, >> because we need to bake the next major - there's too many changes to throw >> out an alpha/beta/rc and hope someone actually runs it. >> >> I don't believe Q3/Q4 is realistic, but I may be biased (or jaded). It's >> possible Q3/Q4 alpha/beta is realistic, but definitely not a release. >> >> >> >> >> On Sun, Feb 11, 2018 at 8:29 PM, kurt greaves <k...@instaclustr.com> >> wrote: >> >>> Hi friends, >>> *TL;DR: Making a plan for 4.0, ideally everyone interested should >>> provide up to two lists, one for tickets they can contribute resources to >>> getting finished, and one for features they think would be desirable for >>> 4.0, but not necessarily have the resources to commit to helping with.* >>> >>> So we had that Roadmap for 4.0 discussion last year, but there was never >>> a conclusion or a plan that came from it. Times getting on and the changes >>> list for 4.0 is getting pretty big. I'm thinking it would probably make >>> sense to define some goals to getting 4.0 released/have an actual plan. 4.0 >>> is already going to be quite an unwieldy release with a lot of testing >>> required. >>> >>> Note: the following is open to discussion, if people don't like the plan >>> feel free to speak up. But in the end it's a pretty basic plan and I don't >>> think we should over-complicate it, I also don't want to end up in a >>> discussion where we "make a plan to make a plan". Regardless of whatever >>> plan we do end up following it would still be valuable to have a list of >>> tickets for 4.0 which is the overall goal of this email - so let's not get >>> too worked up on the details just yet (save that for after I >>> summarise/follow up). >>> >>> // TODO >>> I think the best way to go about this would be for us to come up with a >>> list of JIRA's that we want included in 4.0, tag these as 4.0, and all >>> other improvements as 4.x. We can then aim to release 4.0 once all the 4.0 >>> tagged tickets (+bug fixes/blockers) are complete. >>> >>> Now, the catch is that we obviously don't want to include too many >>> tickets in 4.0, but at the same time we want to make sure 4.0 has an >>> appealing feature set for both users/operators/developers. To minimise >>> scope creep I think the following strategy will help: >>> >>> We should maintain two lists: >>> >>> 1. JIRA's that people want in 4.0 and can commit resources to >>> getting them implemented in 4.0. >>> 2. JIRA's that people simply think would be desirable for 4.0, but >>> currently don't have anyone assigned to them or planned assignment. It >>> would probably make sense to label these with an additional tag in JIRA. >>> *(User's >>> please feel free to point out what you want here)* >>> >>> From list 1 will come our source of truth for when we release 4.0. >>> (after aggregating a list I will summarise and we can vote on it). >>> >>> List 2 would be the "hopeful" list, where stories can be picked up from >>> if resourcing allows, or where someone comes along and decides it's good >>> enough to work on. I guess we can also base this on a vote system if we >>> reach the point of including some of them. (but for the moment it's purely >>> to get an idea of what users actually want). >>> >>> Please don't refrain from listing something that's already been >>> mentioned. The purpose is to get an idea of everyone's priorities/interests >>> and the resources available. We will need multiple resources for each >>> ticket, so anywhere we share an interest will make for a lot easier work >>> sharing. >>> >>> Note that we are only talking about improvements here. Bugs will be >>> treated the same as always, and major issues/regressions we'll need to fix >>> prior to 4.0 anyway. >>> >>> TIME FRAME >>> Generally I think it's a bad idea to commit to any hard deadline, but we >>> should have some time frames in mind. My idea would be to aim for a Q3/4 >>> 2018 release, and as we go we just review the outstanding improvements and >>> decide whether it's worth pushing it back or if we've got enough to >>> release. I suppose keep this time frame in mind when choosing your tickets. >>> >>> We can aim for an earlier date (midyear?) but I figure the >>> testing/validation/bugfixing period prior to release might drag on a bit so >>> being a bit conservative here. >>> The main goal would be to not let list 1 grow unless we're well ahead, >>> and only cull from it if we're heavily over-committed or we decide the >>> improvement can wait. I assume this all sounds like common sense but >>> figured it's better to spell it out now. >>> >>> >>> NEXT STEPS >>> After 2 weeks/whenever the discussion dies off I'll consolidate all the >>> tickets, relevant comments and follow up with a summary, where we can >>> discuss/nitpick issues and come up with a final list to go ahead with. >>> >>> On a side note, in conjunction with this effort we'll obviously have to >>> do something about validation and testing. I'll keep that out of this email >>> for now, but there will be a follow up so that those of us willing to help >>> validate/test trunk can avoid duplicating effort. >>> >>> REVIEW >>> This is the list of "huge/breaking" tickets that got mentioned in the >>> last roadmap discussion and their statuses. This is not terribly important >>> but just so we can keep in mind what we previously talked about. I think we >>> leave it up to the relevant contributors to decide whether they want to get >>> the still open tickets into 4.0. >>> >>> CASSANDRA-9425 Immutable node-local schema >>> <https://issues.apache.org/jira/browse/CASSANDRA-9425> - Committed >>> CASSANDRA-10699 Strongly consistent schema alterations >>> <https://issues.apache.org/jira/browse/CASSANDRA-10699> - Open, no >>> discussion in quite some time. >>> CASSANDRA-12229 NIO streaming >>> <https://issues.apache.org/jira/browse/CASSANDRA-12229> - Committed >>> CASSANDRA-8457 NIO messaging >>> <https://issues.apache.org/jira/browse/CASSANDRA-8457> - Committed >>> CASSANDRA-12345 Gossip 2.0 >>> <https://issues.apache.org/jira/browse/CASSANDRA-12345> - Open, no sign >>> of any action. >>> CASSANDRA-9754 Make index info heap friendly for large CQL partitions >>> <https://issues.apache.org/jira/browse/CASSANDRA-9754> - In progress >>> but no update in a long time. >>> CASSANDRA-11559 enhanced node representation >>> <https://issues.apache.org/jira/browse/CASSANDRA-11559> - Open, no >>> change since early 2016. >>> CASSANDRA-6246 epaxos >>> <https://issues.apache.org/jira/browse/CASSANDRA-6246> - In progress >>> but no update since Feb 2017. >>> CASSANDRA-7544 storage port configurable per node >>> <https://issues.apache.org/jira/browse/CASSANDRA-7544> - Committed >>> CASSANDRA-11115 remove thrift support >>> <https://issues.apache.org/jira/browse/CASSANDRA-11115> - Committed >>> CASSANDRA-10857 dropping compact storage >>> <https://issues.apache.org/jira/browse/CASSANDRA-10857> - Committed >>> >>> To start us off... >>> And here are my lists to get us started. >>> 1. >>> CASSANDRA-8460 - Tiered/Cold storage for TWCS >>> <https://issues.apache.org/jira/browse/CASSANDRA-8460> >>> CASSANDRA-12783 - Batchlog redesign >>> <https://issues.apache.org/jira/browse/CASSANDRA-12783> >>> CASSANDRA-11559 - Enchance node representation >>> <https://issues.apache.org/jira/browse/CASSANDRA-11559> >>> CASSANDRA-12344 - Forward writes to replacement node with same >>> address <https://issues.apache.org/jira/browse/CASSANDRA-12344> >>> CASSANDRA-8119 - More expressive Consistency Levels >>> <https://issues.apache.org/jira/browse/CASSANDRA-8119> >>> CASSANDRA-14210 - Optimise SSTables upgrade task scheduling >>> <https://issues.apache.org/jira/browse/CASSANDRA-14210> >>> CASSANDRA-10540 - RangeAwareCompaction >>> <https://issues.apache.org/jira/browse/CASSANDRA-10540> >>> >>> >>> 2: >>> CASSANDRA-10726 - Read repair inserts should not be blocking >>> <https://issues.apache.org/jira/browse/CASSANDRA-10726> >>> CASSANDRA-9754 - Make index info heap friendly for large CQL partitions >>> <https://issues.apache.org/jira/browse/CASSANDRA-9754> >>> CASSANDRA-12294 - LDAP auth >>> <https://issues.apache.org/jira/browse/CASSANDRA-12294> >>> CASSANDRA-12151 - Audit logging >>> <https://issues.apache.org/jira/browse/CASSANDRA-12151> >>> CASSANDRA-10495 - Fix streaming with vnodes >>> <https://issues.apache.org/jira/browse/CASSANDRA-10495> >>> >>> Also, here's some handy JQL to start you off: >>> project = CASSANDRA AND fixVersion in (4.x, 4.0) AND issue in >>> watchedIssues() AND status != Resolved >>> >>> >> > -- Ben Bromhead CTO | Instaclustr <https://www.instaclustr.com/> +1 650 284 9692 Reliability at Scale Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer
