For what it's worth (and based on the project experience), I think the strategy of "let's agree on a list of tickets everyone would love to get in before we freeze 4.0" doesn't work very well (it's largely useless, expect for making us feel good about not releasing anything). Those lists always end up being too big especially given we have no control on people's ability to contribute (some stuffs will always lag for a very long time, even when they sound really cool on paper).
I'm also a bit sad that we seem to be getting back to our old demons of trying to shove as much as we possibly can in the next major as if having a feature miss it means it will never happen. The 4.0 changelog is big already and we haven't made a release with new features in almost a year now, so I personally think we should start being a bit more aggressive with it and learn to get comfortable letting feature slip if they are not ready. My concrete proposal would be to declare a feature freeze for 4.0 in 2 months, so say June 1th. That leave some time for finishing features that are in progress, but not too much to get derailed. And let's be strict on that freeze. After that, we'll see how quickly we can get stuffs to stabilize but I'd suggest aiming for an alpha 3-4 weeks after that. Of course, we should probably (re-(re-(re-)))start a discussion on release "strategy" in parallel because it doesn't seem we have one right now, but that's imo a discussion we should keep separate. -- Sylvain On Mon, Apr 2, 2018 at 4:54 PM DuyHai Doan <doanduy...@gmail.com> wrote: > My wish list: > > * Add support for arithmetic operators (CASSANDRA-11935) > * Allow IN restrictions on column families with collections > (CASSANDRA-12654) > * Add support for + and - operations on dates (CASSANDRA-11936) > * Add the currentTimestamp, currentDate, currentTime and currentTimeUUID > functions (CASSANDRA-13132) > * Allow selecting Map values and Set elements (CASSANDRA-7396) > > Those are mostly useful for timeseries data models and I guess has no > significant impact on the internals and operations so the risk of > regression is low > > On Mon, Apr 2, 2018 at 4:33 PM, Jeff Jirsa <jji...@gmail.com> wrote: > > > 9608 (java9) > > > > -- > > Jeff Jirsa > > > > > > > On Apr 2, 2018, at 3:45 AM, Jason Brown <jasedbr...@gmail.com> wrote: > > > > > > The only additional tickets I'd like to mention are: > > > > > > https://issues.apache.org/jira/browse/CASSANDRA-13971 - Automatic > > > certificate management using Vault > > > - Stefan's Vault integration work. A sub-ticket, CASSANDRA-14102, > > addresses > > > encryption at-rest, subsumes CASSANDRA-9633 (SSTable encryption) - > which > > I > > > doubt I would be able to get to any time this year. It would definitely > > be > > > nice to have a clarified encryption/security story for 4.0. > > > > > > https://issues.apache.org/jira/browse/CASSANDRA-11990 - Address rows > > rather > > > than partitions in SASI > > > - a nice update for SASI, but not critical. > > > > > > -Jason > > > > > >> On Sat, Mar 31, 2018 at 6:53 PM, Ben Bromhead <b...@instaclustr.com> > > wrote: > > >> > > >> Apologies all, I didn't realize I was responding to this discussion > > only on > > >> the @user list. One of the perils of responding to a thread that is on > > both > > >> user and dev... > > >> > > >> For context, I have included my response to Kurt's previous discussion > > on > > >> this topic as it only ended up on the user list. > > >> > > >> *After some further discussions with folks offline, I'd like to revive > > this > > >> discussion. * > > >> > > >> *As Kurt mentioned, to keep it simple I if we can simply build > consensus > > >> around what is in for 4.0 and what is out. We can then start the > > process of > > >> working off a 4.0 branch towards betas and release candidates. Again > as > > >> Kurt mentioned, assigning a timeline to it right now is difficult, but > > >> having a firm line in the sand around what features/patches are in, > then > > >> limiting future 4.0 work to bug fixes will give folks a less nebulous > > >> target to work on. * > > >> > > >> *The other thing to mention is that once we have a 4.0 branch to work > > off, > > >> we at Instaclustr have a commitment to dogfooding the release > > candidates on > > >> our internal staging and internal production workloads before 4.0 > > becomes > > >> generally available. I know other folks have similar commitments and > > simply > > >> having a 4.0 branch with a clear list of things that are in or out > will > > >> allow everyone to start testing and driving towards a quality > release. * > > >> > > >> *The other thing is that there are already a large number of changes > > ready > > >> for 4.0, I would suggest not recommending tickets for 4.0 that have > not > > yet > > >> been finished/have outstanding work unless you are the person working > > on it > > >> (or are offering to work on it instead) and can get it ready for > review > > in > > >> a timely fashion. That way we can build a more realistic working > target. > > >> For other major breaking changes, there is always 5.0 or 4.1 or > > whatever we > > >> end up doing :)* > > >> > > >> Thinking further about it, I would suggest a similar process that was > > >> applied to releasing 3.0, in order to get to 4.0: > > >> > > >> - Clean up ticket labeling. Move tickets unlikely to make it / be > > worked > > >> on for 4.0 to something else (e.g. 4.x or whatever). > > >> - Tickets labeled 4.0 will be the line in the sand, with some > trigger > > >> ("done") event where all features not done by a certain event will > > >> simply > > >> move into the next release. For the 3.0 branch, this occurred after > a > > >> large review of 8099. For 4.0 it could simply be resolving all > current > > >> blockers/major tickets tagged 4.0... doesn't have to be / nor is it > > >> something I would strongly advocate. > > >> - Once we hit this "done" event. Cut a Cassandra-4.0 branch and > start > > >> the alpha/beta/rc cycle from that branch, with only bugfixes going > > into > > >> it > > >> - This, in my mind, is similar to the 3.0 approach > > >> https://mail-archives.apache.org/mod_mbox/cassandra-dev/ > > >> 201503.mbox/%3CCALdd-zjAyiTbZksMeq2LxGwLF5LPhoi_ > > >> 4vsjy8JBHBRnsxH%3D8A%40mail.gmail.com%3E, > > >> but without the subsequent tick-tock :) > > >> > > >> There are currently 3 open blockers tagged 4.0, some are old and > > probably > > >> not really blockers anymore, there are other tickets that may/should > be > > >> blockers on 4.0: > > >> > > >> - https://issues.apache.org/jira/browse/CASSANDRA-13951 > > >> - https://issues.apache.org/jira/browse/CASSANDRA-13994 > > >> - https://issues.apache.org/jira/browse/CASSANDRA-12042 > > >> > > >> In terms of major tickets that I would like to see land: > > >> > > >> - https://issues.apache.org/jira/browse/CASSANDRA-7622 Virtual > Tables > > >> - https://issues.apache.org/jira/browse/CASSANDRA-13628 Internode > > netty > > >> - https://issues.apache.org/jira/browse/CASSANDRA-13475 Pluggable > > >> Storage > > >> - https://issues.apache.org/jira/browse/CASSANDRA-9633 SSTable > > >> encryption > > >> > > >> Ben > > >> > > >>> On Mon, Feb 12, 2018 at 11:26 PM Jeff Jirsa <jji...@gmail.com> > wrote: > > >>> > > >>> > > >>> Advantages of cutting a release sooner than later: > > >>> 1) The project needs to constantly progress forward. Releases are the > > >> most > > >>> visible part of that. > > >>> 2) Having a huge changelog in a release increases the likelihood of > > bugs > > >>> that take time to find. > > >>> > > >>> Advantages of a slower release: > > >>> 1) We don't do major versions often, and when we do breaking changes > > >>> (protocol, file format, etc), we should squeeze in as many as > possible > > to > > >>> avoid having to roll new majors > > >>> 2) There are probably few people actually running 3.11 at scale, so > > >>> probably few people actually testing trunk. > > >>> > > >>> In terms of "big" changes I'd like to see land, the ones that come to > > >> mind > > >>> are: > > >>> > > >>> https://issues.apache.org/jira/browse/CASSANDRA-9754 - "Birch" > > (changes > > >>> file format) > > >>> https://issues.apache.org/jira/browse/CASSANDRA-13442 - Transient > > >>> Replicas (probably adds new replication strategy or similar) > > >>> https://issues.apache.org/jira/browse/CASSANDRA-13628 - Rest of the > > >>> internode netty stuff (no idea if this changes internode stuff, but I > > bet > > >>> it's a lot easier if it lands on a major) > > >>> https://issues.apache.org/jira/browse/CASSANDRA-7622 - Virtual > Tables > > >>> (selfish inclusion, probably doesn't need to be a major at all, and I > > >>> wouldn't even lose sleep if it slips, but I'd like to see it land) > > >>> > > >>> Stuff I'm ok with slipping to 4.X or 5.0, but probably needs to land > > on a > > >>> major because we'll change something big (like gossip, or the way > > schema > > >> is > > >>> passed, etc): > > >>> > > >>> https://issues.apache.org/jira/browse/CASSANDRA-9667 - Strongly > > >>> consistent membership > > >>> https://issues.apache.org/jira/browse/CASSANDRA-10699 - Strongly > > >>> consistent schema > > >>> > > >>> All that said, what I really care about is building confidence in the > > >>> release, which means an extended testing cycle. If all of those > patches > > >>> landed tomorrow, I'd still expect us to be months away from a > release, > > >>> because we need to bake the next major - there's too many changes to > > >> throw > > >>> out an alpha/beta/rc and hope someone actually runs it. > > >>> > > >>> I don't believe Q3/Q4 is realistic, but I may be biased (or jaded). > > It's > > >>> possible Q3/Q4 alpha/beta is realistic, but definitely not a release. > > >>> > > >>> > > >>> > > >>> > > >>> On Sun, Feb 11, 2018 at 8:29 PM, kurt greaves <k...@instaclustr.com> > > >>> wrote: > > >>> > > >>>> Hi friends, > > >>>> *TL;DR: Making a plan for 4.0, ideally everyone interested should > > >> provide > > >>>> up to two lists, one for tickets they can contribute resources to > > >> getting > > >>>> finished, and one for features they think would be desirable for > 4.0, > > >> but > > >>>> not necessarily have the resources to commit to helping with.* > > >>>> > > >>>> So we had that Roadmap for 4.0 discussion last year, but there was > > never > > >>>> a conclusion or a plan that came from it. Times getting on and the > > >> changes > > >>>> list for 4.0 is getting pretty big. I'm thinking it would probably > > make > > >>>> sense to define some goals to getting 4.0 released/have an actual > > plan. > > >> 4.0 > > >>>> is already going to be quite an unwieldy release with a lot of > testing > > >>>> required. > > >>>> > > >>>> Note: the following is open to discussion, if people don't like the > > plan > > >>>> feel free to speak up. But in the end it's a pretty basic plan and I > > >> don't > > >>>> think we should over-complicate it, I also don't want to end up in a > > >>>> discussion where we "make a plan to make a plan". Regardless of > > whatever > > >>>> plan we do end up following it would still be valuable to have a > list > > of > > >>>> tickets for 4.0 which is the overall goal of this email - so let's > not > > >> get > > >>>> too worked up on the details just yet (save that for after I > > >>>> summarise/follow up). > > >>>> > > >>>> // TODO > > >>>> I think the best way to go about this would be for us to come up > with > > a > > >>>> list of JIRA's that we want included in 4.0, tag these as 4.0, and > all > > >>>> other improvements as 4.x. We can then aim to release 4.0 once all > the > > >> 4.0 > > >>>> tagged tickets (+bug fixes/blockers) are complete. > > >>>> > > >>>> Now, the catch is that we obviously don't want to include too many > > >>>> tickets in 4.0, but at the same time we want to make sure 4.0 has an > > >>>> appealing feature set for both users/operators/developers. To > minimise > > >>>> scope creep I think the following strategy will help: > > >>>> > > >>>> We should maintain two lists: > > >>>> > > >>>> 1. JIRA's that people want in 4.0 and can commit resources to > > getting > > >>>> them implemented in 4.0. > > >>>> 2. JIRA's that people simply think would be desirable for 4.0, but > > >>>> currently don't have anyone assigned to them or planned > assignment. > > >> It > > >>>> would probably make sense to label these with an additional tag in > > >> JIRA. *(User's > > >>>> please feel free to point out what you want here)* > > >>>> > > >>>> From list 1 will come our source of truth for when we release 4.0. > > >> (after > > >>>> aggregating a list I will summarise and we can vote on it). > > >>>> > > >>>> List 2 would be the "hopeful" list, where stories can be picked up > > from > > >>>> if resourcing allows, or where someone comes along and decides it's > > good > > >>>> enough to work on. I guess we can also base this on a vote system if > > we > > >>>> reach the point of including some of them. (but for the moment it's > > >> purely > > >>>> to get an idea of what users actually want). > > >>>> > > >>>> Please don't refrain from listing something that's already been > > >>>> mentioned. The purpose is to get an idea of everyone's > > >> priorities/interests > > >>>> and the resources available. We will need multiple resources for > each > > >>>> ticket, so anywhere we share an interest will make for a lot easier > > work > > >>>> sharing. > > >>>> > > >>>> Note that we are only talking about improvements here. Bugs will be > > >>>> treated the same as always, and major issues/regressions we'll need > to > > >> fix > > >>>> prior to 4.0 anyway. > > >>>> > > >>>> TIME FRAME > > >>>> Generally I think it's a bad idea to commit to any hard deadline, > but > > we > > >>>> should have some time frames in mind. My idea would be to aim for a > > Q3/4 > > >>>> 2018 release, and as we go we just review the outstanding > improvements > > >> and > > >>>> decide whether it's worth pushing it back or if we've got enough to > > >>>> release. I suppose keep this time frame in mind when choosing your > > >> tickets. > > >>>> > > >>>> We can aim for an earlier date (midyear?) but I figure the > > >>>> testing/validation/bugfixing period prior to release might drag on a > > >> bit so > > >>>> being a bit conservative here. > > >>>> The main goal would be to not let list 1 grow unless we're well > ahead, > > >>>> and only cull from it if we're heavily over-committed or we decide > the > > >>>> improvement can wait. I assume this all sounds like common sense but > > >>>> figured it's better to spell it out now. > > >>>> > > >>>> > > >>>> NEXT STEPS > > >>>> After 2 weeks/whenever the discussion dies off I'll consolidate all > > the > > >>>> tickets, relevant comments and follow up with a summary, where we > can > > >>>> discuss/nitpick issues and come up with a final list to go ahead > with. > > >>>> > > >>>> On a side note, in conjunction with this effort we'll obviously have > > to > > >>>> do something about validation and testing. I'll keep that out of > this > > >> email > > >>>> for now, but there will be a follow up so that those of us willing > to > > >> help > > >>>> validate/test trunk can avoid duplicating effort. > > >>>> > > >>>> REVIEW > > >>>> This is the list of "huge/breaking" tickets that got mentioned in > the > > >>>> last roadmap discussion and their statuses. This is not terribly > > >> important > > >>>> but just so we can keep in mind what we previously talked about. I > > >> think we > > >>>> leave it up to the relevant contributors to decide whether they want > > to > > >> get > > >>>> the still open tickets into 4.0. > > >>>> > > >>>> CASSANDRA-9425 Immutable node-local schema > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-9425> - Committed > > >>>> CASSANDRA-10699 Strongly consistent schema alterations > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-10699> - Open, no > > >>>> discussion in quite some time. > > >>>> CASSANDRA-12229 NIO streaming > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-12229> - Committed > > >>>> CASSANDRA-8457 NIO messaging > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-8457> - Committed > > >>>> CASSANDRA-12345 Gossip 2.0 > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-12345> - Open, no > > sign > > >>>> of any action. > > >>>> CASSANDRA-9754 Make index info heap friendly for large CQL > partitions > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-9754> - In > progress > > >> but > > >>>> no update in a long time. > > >>>> CASSANDRA-11559 enhanced node representation > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-11559> - Open, no > > >>>> change since early 2016. > > >>>> CASSANDRA-6246 epaxos > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-6246> - In > progress > > >> but > > >>>> no update since Feb 2017. > > >>>> CASSANDRA-7544 storage port configurable per node > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-7544> - Committed > > >>>> CASSANDRA-11115 remove thrift support > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-11115> - Committed > > >>>> CASSANDRA-10857 dropping compact storage > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-10857> - Committed > > >>>> > > >>>> To start us off... > > >>>> And here are my lists to get us started. > > >>>> 1. > > >>>> CASSANDRA-8460 - Tiered/Cold storage for TWCS > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-8460> > > >>>> CASSANDRA-12783 - Batchlog redesign > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-12783> > > >>>> CASSANDRA-11559 - Enchance node representation > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-11559> > > >>>> CASSANDRA-12344 - Forward writes to replacement node with same > > >>>> address <https://issues.apache.org/jira/browse/CASSANDRA-12344> > > >>>> CASSANDRA-8119 - More expressive Consistency Levels > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-8119> > > >>>> CASSANDRA-14210 - Optimise SSTables upgrade task scheduling > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-14210> > > >>>> CASSANDRA-10540 - RangeAwareCompaction > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-10540> > > >>>> > > >>>> > > >>>> 2: > > >>>> CASSANDRA-10726 - Read repair inserts should not be blocking > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-10726> > > >>>> CASSANDRA-9754 - Make index info heap friendly for large CQL > > partitions > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-9754> > > >>>> CASSANDRA-12294 - LDAP auth > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-12294> > > >>>> CASSANDRA-12151 - Audit logging > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-12151> > > >>>> CASSANDRA-10495 - Fix streaming with vnodes > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-10495> > > >>>> > > >>>> Also, here's some handy JQL to start you off: > > >>>> project = CASSANDRA AND fixVersion in (4.x, 4.0) AND issue in > > >>>> watchedIssues() AND status != Resolved > > >>>> > > >>>> > > >>> -- > > >> Ben Bromhead > > >> CTO | Instaclustr <https://www.instaclustr.com/> > > >> +1 650 284 9692 > > >> Reliability at Scale > > >> Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer > > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > >
