Thanks for volunteering, Jason! On Wed, Sep 7, 2016 at 1:59 AM Ismael Juma <ism...@juma.me.uk> wrote:
> Thanks for volunteering Jason. Sounds good to me, > > Ismael > > On Wed, Sep 7, 2016 at 4:22 AM, Jason Gustafson <ja...@confluent.io> > wrote: > > > Hey All, > > > > It sounds like the general consensus is in favor of time-based releases. > We > > can continue the discussion about LTS, but I wanted to go ahead and get > > things moving forward by volunteering to manage the next release, which > is > > currently slated for October. If that sounds OK, I'll draft a release > plan > > and send it out to the community for feedback and a vote. > > > > Thanks, > > Jason > > > > On Thu, Aug 25, 2016 at 2:03 PM, Ofir Manor <ofir.ma...@equalum.io> > wrote: > > > > > I happily agree that Kafka is a solid and the community is great :) > > > But I think there is a gap in perception here. > > > For me, LTS means that someone is actively taking care of a release - > > > actively backporting critical fixes (security, stability, data loss, > > > corruption, hangs etc) from trunk to that LTS version periodically for > an > > > extended period of time, for example 18-36 months... So people can > really > > > rely on the same Kafka version for a long time. > > > Is someone doing it today for 0.9.0? When is 0.9.0.2 expected? When is > > > 0.8.2.3 expected? Will they cover all known critical issues for whoever > > > relies on them in production? > > > In other words, what is the scope of support that the community want to > > > commit for older versions? (upgrade compatibility? investigating bug > > > reports? proactively backporting fixes?) > > > BTW, another legit option is that the Apache Kafka project won't commit > > to > > > LTS releases. It could let commercial vendors compete on supporting > very > > > old versions. I find that actually quite reasonable as well. > > > > > > Ofir Manor > > > > > > Co-Founder & CTO | Equalum > > > > > > Mobile: +972-54-7801286 | Email: ofir.ma...@equalum.io > > > > > > On Thu, Aug 25, 2016 at 8:19 PM, Andrew Schofield < > > > andrew_schofield_j...@outlook.com> wrote: > > > > > > > I agree that the Kafka community has managed to maintain a very high > > > > quality level, so I'm not concerned > > > > about the quality of non-LTS releases. If the principle is that every > > > > release is supported for 2 years, that > > > > would be good. I suppose that if the burden of having that many > > > in-support > > > > releases proves too heavy, > > > > as you say we could reconsider. > > > > > > > > Andrew Schofield > > > > > > > > ---------------------------------------- > > > > > From: g...@confluent.io > > > > > Date: Thu, 25 Aug 2016 09:57:30 -0700 > > > > > Subject: Re: [DISCUSS] Time-based releases for Apache Kafka > > > > > To: dev@kafka.apache.org > > > > > > > > > > I prefer Ismael's suggestion for supporting 2-years (6 releases) > > > > > rather than have designated LTS releases. > > > > > > > > > > The LTS model seems to work well when some releases are high > quality > > > > > (LTS) and the rest are a bit more questionable. It is great for > > > > > companies like Redhat, where they have to invest less to support > few > > > > > releases and let the community deal with everything else. > > > > > > > > > > Until now the Kafka community has managed to maintain very high > > > > > quality level. Not just for releases, our trunk is often of better > > > > > quality than other project's releases - we don't think of stability > > as > > > > > something you tuck into a release (and just some releases) but > rather > > > > > as an on-going concern. There are costs to doing things that way, > but > > > > > in general, I think it has served us well - allowing even > > conservative > > > > > companies to run on the latest released version. > > > > > > > > > > I hope we can agree to at least try maintaining last 6 releases as > > LTS > > > > > (i.e. every single release is supported for 2 years) rather than > > > > > designate some releases as better than others. Of course, if this > > > > > totally fails, we can reconsider. > > > > > > > > > > Gwen > > > > > > > > > > On Thu, Aug 25, 2016 at 9:51 AM, Andrew Schofield > > > > > <andrew_schofield_j...@outlook.com> wrote: > > > > >> The proposal sounds pretty good, but the main thing currently > > missing > > > > is a proper long-term support release. > > > > >> > > > > >> Having 3 releases a year sounds OK, but if they're all equivalent > > and > > > > bugfix releases are produced for the most > > > > >> recent 2 or 3 releases, anyone wanting to run on an "in support" > > > > release of Kafka has to upgrade every 8-12 months. > > > > >> If you don't actually want anything specific from the newer > > releases, > > > > it's just unnecessary churn. > > > > >> > > > > >> Wouldn't it be better to designate one release every 12-18 months > > as a > > > > long-term support release with bugfix releases > > > > >> produced for those for a longer period of say 24 months. That > halves > > > > the upgrade work for people just wanting to keep > > > > >> "in support". Now that adoption is increasing, there are plenty of > > > > users that just want a dependable messaging system > > > > >> without having to be deeply knowledgeable about its innards. > > > > >> > > > > >> LTS works nicely for plenty of open-source projects. I think it > > would > > > > work well for Kafka too. > > > > >> > > > > >> Andrew Schofield > > > > >> > > > > >> ---------------------------------------- > > > > >>> From: ofir.ma...@equalum.io > > > > >>> Date: Thu, 25 Aug 2016 16:07:07 +0300 > > > > >>> Subject: Re: [DISCUSS] Time-based releases for Apache Kafka > > > > >>> To: dev@kafka.apache.org > > > > >>> > > > > >>> Regarding bug fixes, you may want to consider to have an LTS > > release > > > > once a > > > > >>> year - designating it for longer-term support / better for the > > > masses. > > > > >>> If you like that - then fix bugs in trunk, backport important > ones > > to > > > > >>> latest release + the last two LTS releases. > > > > >>> Even if you don't - if a downstream distribution picks a Kafka > > > version > > > > and > > > > >>> plans to support it over a few years, it could be nice of them to > > > "own" > > > > >>> that older release - volunteer to be a release manager for bug > > > > backports to > > > > >>> that version over a longer period... > > > > >>> Just my two cents :) > > > > >>> > > > > >>> Ofir Manor > > > > >>> > > > > >>> Co-Founder & CTO | Equalum > > > > >>> > > > > >>> Mobile: +972-54-7801286 | Email: ofir.ma...@equalum.io > > > > >>> > > > > >>> On Thu, Aug 25, 2016 at 12:32 PM, Ismael Juma <ism...@juma.me.uk > > > > > > wrote: > > > > >>> > > > > >>>> Thanks for putting this together Gwen. I think it sounds > > reasonable > > > > and > > > > >>>> instead of trying to optimise every aspect of it ahead of time > > > (which > > > > is > > > > >>>> hard, subjective and time-consuming), I am happy to try what is > > > being > > > > >>>> proposed and tweak based on experience. One thing we should pay > > > > particular > > > > >>>> attention to is how the stabilisation process works out in > > practice. > > > > >>>> > > > > >>>> A couple of comments: > > > > >>>> > > > > >>>> "Given 3 releases a year and the fact that no one upgrades three > > > > times a > > > > >>>> year, we propose making sure (by testing!) that rolling upgrade > > can > > > > be done > > > > >>>> from each release in the past year (i.e. last 3 releases) to the > > > > latest > > > > >>>> version." > > > > >>>> > > > > >>>> Because the cost of doing this for a larger number of releases > is > > > > >>>> relatively low, I still think we should go for 6 here (our code > > > > currently > > > > >>>> supports 5 versions as I said in a previous message, so we're > > close > > > > to that > > > > >>>> target already). I'm generally very keen to make upgrades as > easy > > as > > > > >>>> possible so that people have no reason not to upgrade. :) > > > > >>>> > > > > >>>> "We will also attempt, as a community to do bugfix releases as > > > needed > > > > for > > > > >>>> the last 3 releases." > > > > >>>> > > > > >>>> I would suggest 2, personally, but since this is a bit fuzzy, I > am > > > OK > > > > with > > > > >>>> 3 if people prefer that. > > > > >>>> > > > > >>>> Ismael > > > > >>>> > > > > >>>> On Thu, Aug 25, 2016 at 6:22 AM, Gwen Shapira < > g...@confluent.io> > > > > wrote: > > > > >>>> > > > > >>>>> Hi Team Kafka, > > > > >>>>> > > > > >>>>> As per the KIP meeting, I created a wiki: > > > > >>>>> https://cwiki.apache.org/confluence/display/KAFKA/Time+ > > > > >>>> Based+Release+Plan > > > > >>>>> Summarizing most of the discussion so far. > > > > >>>>> > > > > >>>>> Comments and additional discussion is welcome :) > > > > >>>>> > > > > >>>>> Gwen > > > > >>>>> > > > > >>>>> On Wed, Aug 17, 2016 at 12:31 PM, Vahid S Hashemian > > > > >>>>> <vahidhashem...@us.ibm.com> wrote: > > > > >>>>>> Time-based releases is a good idea and something that has > proved > > > to > > > > be > > > > >>>>>> working in a number of open source projects. One successful > > > example > > > > is > > > > >>>>>> Node.js, that goes through two major releases a year. The > > > > interesting > > > > >>>>> fact > > > > >>>>>> about the two releases is that only one (the even-number > > release) > > > > comes > > > > >>>>>> with a long term support (LTS) plan (30 months). More can be > > read > > > > here: > > > > >>>>>> https://github.com/nodejs/LTS. The odd-number releases still > > come > > > > with > > > > >>>>>> major changes and help build the ecosystem, but as far as LTS > > > goes, > > > > >>>> there > > > > >>>>>> is only one per year. This LTS plan makes most enterprises > want > > to > > > > >>>> stick > > > > >>>>>> to even-number releases, which is okay since frequent upgrades > > is > > > > not > > > > >>>>>> something they are normally interested in anyway. > > > > >>>>>> > > > > >>>>>> There could be several minor releases (non-breaking) in > between > > > > major > > > > >>>>>> releases. A major release contains all the features / bug > fixes > > in > > > > the > > > > >>>>>> master branch a month before the release date, with the > > potential > > > > >>>>> addition > > > > >>>>>> of (non-breaking) bug fixes until the release day. The > > deprecation > > > > >>>> cycle > > > > >>>>>> is one major release: any functionality that is decided to be > > > > removed > > > > >>>> is > > > > >>>>>> deprecated in the next major release, and removed in the major > > > > release > > > > >>>>>> after that. > > > > >>>>>> > > > > >>>>>> Because of the success of LTS models in this and other open > > source > > > > >>>>>> projects, I would suggest implementing a formal LTS plan for > > Kafka > > > > too. > > > > >>>>>> > > > > >>>>>> Regards, > > > > >>>>>> --Vahid > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> From: Gwen Shapira <g...@confluent.io> > > > > >>>>>> To: dev@kafka.apache.org > > > > >>>>>> Date: 08/09/2016 04:49 PM > > > > >>>>>> Subject: [DISCUSS] Time-based releases for Apache Kafka > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> Dear Kafka Developers and Users, > > > > >>>>>> > > > > >>>>>> In the past, our releases have been quite unpredictable. We'll > > > > notice > > > > >>>>>> that a large number of nice features made it in (or are > close), > > > > >>>>>> someone would suggest a release and we'd do it. This is fun, > but > > > > makes > > > > >>>>>> planning really hard - we saw it during the last release which > > we > > > > >>>>>> decided to delay by a few weeks to allow more features to > > "land". > > > > >>>>>> > > > > >>>>>> Many other communities have adopted time-based releases > > > successfully > > > > >>>>>> (Cassandra, GCC, LLVM, Fedora, Gnome, Ubuntu, etc.). And I > > thought > > > > it > > > > >>>>>> will make sense for the Apache Kafka community to try doing > the > > > > same. > > > > >>>>>> > > > > >>>>>> The benefits of this approach are: > > > > >>>>>> > > > > >>>>>> 1. A quicker feedback cycle and users can benefit from > features > > > > >>>>>> quicker (assuming for reasonably short time between releases > - I > > > was > > > > >>>>>> thinking 4 months) > > > > >>>>>> > > > > >>>>>> 2. Predictability for contributors and users: > > > > >>>>>> * Developers and reviewers can decide in advance what release > > they > > > > are > > > > >>>>>> aiming for with specific features. > > > > >>>>>> * If a feature misses a release we have a good idea of when it > > > will > > > > >>>> show > > > > >>>>>> up. > > > > >>>>>> * Users know when to expect their features > > > > >>>>>> > > > > >>>>>> 3. Transparency - There will be a published cut-off date (AKA > > > > feature > > > > >>>>>> freeze) for the release and people will know about it in > > advance. > > > > >>>>>> Hopefully this will remove the contention around which > features > > > make > > > > >>>>>> it. > > > > >>>>>> > > > > >>>>>> 4. Quality - we've seen issues pop up in release candidates > due > > to > > > > >>>>>> last-minute features that didn't have proper time to bake in. > > More > > > > >>>>>> time between feature freeze and release will let us test more, > > > > >>>>>> document more and resolve more issues. > > > > >>>>>> > > > > >>>>>> Since nothing is ever perfect, there will be some downsides: > > > > >>>>>> > > > > >>>>>> 1. Most notably, features that miss the feature-freeze date > for > > a > > > > >>>>>> release will have to wait few month for the next release. > > Features > > > > >>>>>> will reach users faster overall as per benefit #1, but > > individual > > > > >>>>>> features that just miss the cut will lose out > > > > >>>>>> > > > > >>>>>> 2. More releases a year mean that being a committer is more > > work - > > > > >>>>>> release management is still some headache and we'll have more > of > > > > >>>>>> those. Hopefully we'll get better at it. Also, the committer > > list > > > is > > > > >>>>>> growing and hopefully it will be less than once-a-year effort > > for > > > > each > > > > >>>>>> committer. > > > > >>>>>> > > > > >>>>>> 3. For users, figuring out which release to use and having > > > frequent > > > > >>>>>> new releases to upgrade to may be a bit confusing. > > > > >>>>>> > > > > >>>>>> 4. Frequent releases mean we need to do bugfix releases for > > older > > > > >>>>>> branches. Right now we only do bugfix releases to latest > > release. > > > > >>>>>> > > > > >>>>>> I think the benefits outweigh the drawbacks. Or at least > suggest > > > > that > > > > >>>>>> its worth trying - we can have another discussion in few > > releases > > > to > > > > >>>>>> see if we want to keep it that way or try something else. > > > > >>>>>> > > > > >>>>>> My suggestion for the process: > > > > >>>>>> > > > > >>>>>> 1. We decide on a reasonable release cadence > > > > >>>>>> 2. We decide on release dates (even rough estimate such as > "end > > of > > > > >>>>>> February" or something) and work back feature freeze dates. > > > > >>>>>> 3. Committers volunteer to be "release managers" for specific > > > > >>>>>> releases. We can coordinate on the list or on a wiki. If no > > > > committer > > > > >>>>>> volunteers, we assume the community doesn't need a release and > > > skip > > > > >>>>>> it. > > > > >>>>>> 4. At the "feature freeze" date, the release manager announces > > the > > > > >>>>>> contents of the release (which KIPs made it in on time), > creates > > > the > > > > >>>>>> release branch and starts the release process as usual. From > > this > > > > >>>>>> point onwards, only bug fixes should be double-committed to > the > > > > >>>>>> release branch while trunk can start collecting features for > the > > > > >>>>>> subsequent release. > > > > >>>>>> > > > > >>>>>> Comments and improvements are appreciated. > > > > >>>>>> > > > > >>>>>> Gwen Shapira > > > > >>>>>> Former-release-manager > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>> > > > > >>>>> > > > > >>>>> > > > > >>>>> -- > > > > >>>>> Gwen Shapira > > > > >>>>> Product Manager | Confluent > > > > >>>>> 650.450.2760 | @gwenshap > > > > >>>>> Follow us: Twitter | blog > > > > >>>>> > > > > >>>> > > > > >> > > > > > > > > > > > > > > > > > > > > -- > > > > > Gwen Shapira > > > > > Product Manager | Confluent > > > > > 650.450.2760 | @gwenshap > > > > > Follow us: Twitter | blog > > > > > > > > > > > > > > -- Thanks, Neha
