> At the same time, even if something is in Preview and we would guarantee backward compatibility, a major release would still allow us to break it?
I think so. The way I see it is the following. We have 3 levels of graduation: 1, 2 and 3 (naming is hard and don't want to bike shed on this). Level 1 is the first level and the "lower" in terms of production readiness. In this level there are no guarantees, just best of intentions. Compatibility might be broken between minor releases and why not between patches. Maybe the feature is not even complete. Level 2 is one level "up". We need to provide some more guarantees. Probably the same level of compatibility as with any other feature. But what we can't probably assure is any type of SLA/SLO, but the community is encouraged to test in production or staging environments. In plain words: "it should work but we are not completely sure it works under different situations or deployment models" for example. In this level, feature must be complete. Finally, we have level 3, the final one. This is the stage where we are confident that the feature works in all normal conditions (of course bugs can occur). A new feature can go through all 3 levels, or start on number 2 or jump directly to number 3. Also a feature can be killed in level 1. After level 1, a feature should undergo a deprecation process. I think we have enough feedback and general consensus to start drafting a KIP. I'll do in the following days. Best, ------------------ Josep Prat Open Source Engineering Director, Aiven josep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, Anna Richardson, Kenneth Chen Amtsgericht Charlottenburg, HRB 209739 B On Thu, Aug 1, 2024, 23:44 Matthias J. Sax <mj...@apache.org> wrote: > Interesting point. To me, EA could be the same as "experimental". I > don't think we should introduce too many categories. > > From a naming point of view, the difference between EA and Preview is > not super intuitive though. If we would go with "experimental" instead > of EA it might be easier to understand? > > > At the same time, even if something is in Preview and we would guarantee > backward compatibility, a major release would still allow us to break it? > > > > -Matthias > > > On 8/1/24 12:32 PM, Colin McCabe wrote: > > Hi all, > > > > Sorry, I mis-spoke earlier. For KRaft, we did EA -> Preview -> GA. For > some reason I remembered the order differently, but I was wrong. > > > > EA was 2.8; preview was 3.0, GA was 3.3. > > > > The main thing I remember was that we broke compatibility between KRaft > 2.8 and everything else. That release really was just a demo, in some > sense. In the 3.x line, we tried much harder to keep backwards > compatibility (and to the best of my recollection, did). > > > > From a user's point of view, understanding whether compatibility could > be broken in the future is probably the most important thing. Unfortunately > we don't seem to be consistent on this; some EA features moved smoothly to > GA with no compatibility breaks, while others did drop compat. It could be > useful to have a term for features that for which upgrade is not supported. > Perhaps "experimental"? > > > > best, > > Colin > > > > > > On Wed, Jul 31, 2024, at 20:50, Chris Egerton wrote: > >> I only raised the idea of barrier to entry regarding the template. I did > >> not state or even really imply that it should affect this initiative in > >> general. Please do not feel obligated to discuss that concept further if > >> we're all on the same page RE not adding to the template. > >> > >> On Wed, Jul 31, 2024, 22:02 Matthias J. Sax <mj...@apache.org> wrote: > >> > >>> Hi, > >>> > >>> thanks for starting this discussion. It's for sure useful. > >>> > >>> I don't care too much about the naming personally, it just should be > >>> well defined. > >>> > >>> I agree to Josep, that I don't think we would need to change the KIP > >>> template. A KIP is a design doc for the full feature, and should not > >>> concern itself with what state is covered in which release. (As a > matter > >>> of fact, most KIPs don't even include their release version -- we only > >>> track it in the top-level KIP overview page...) > >>> > >>> It might be useful though, to add information about different stages in > >>> the KIP overview page, which atm only says "released in version 3.8" > >>> what seems not to be fined grained enough, and hard to reason about. > For > >>> something like KIP-848 this should be simpler to achieve as it's single > >>> KIP. For KRaft which actually is a high level umbrella KIP with many > >>> child KIPs it's much harder to reason about it though. Maybe the table > >>> we use right now it not the right way for stuff like this and we should > >>> just track this differently (not sure right now how to be fair). > >>> > >>> Keeping entry level low, seems not to be a concern, given that new > >>> contributes would most likely work on smaller KIPs which are completed > >>> in a single release and thus don't require a more complex workflow and > >>> tracking. > >>> > >>> > >>> -Matthias > >>> > >>> On 7/31/24 12:25 PM, Josep Prat wrote: > >>>> Hi, > >>>> > >>>> I think we can keep the KIP template as is and have the extra process > in > >>>> the Jira area. > >>>> > >>>> EA, Preview and Production Ready (I also kind of like this better) > feel > >>> to > >>>> me more project management while the KIP document itself is more about > >>> the > >>>> nature of the changes and describing the end result. Except for some > KIPs > >>>> that inherently describe a process delivered in stages. > >>>> > >>>> Best, > >>>> ------------------ > >>>> Josep Prat > >>>> Open Source Engineering Director, Aiven > >>>> josep.p...@aiven.io | +491715557497 | aiven.io > >>>> Aiven Deutschland GmbH > >>>> Alexanderufer 3-7, 10117 Berlin > >>>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, > >>>> Anna Richardson, Kenneth Chen > >>>> Amtsgericht Charlottenburg, HRB 209739 B > >>>> > >>>> On Wed, Jul 31, 2024, 21:12 Andrew Schofield < > andrew_schofi...@live.com> > >>>> wrote: > >>>> > >>>>> Hi, > >>>>> I would at least like to see agreement on the various stages. I’m > used > >>> to > >>>>> Early Access, then Preview, and finally General Availability, but I > >>> notice > >>>>> that > >>>>> at least one other person on this thread had the order different. > Also, > >>>>> General Availability sounds a bit wrong for an open-source project, > so > >>>>> maybe > >>>>> Production Ready is better. > >>>>> > >>>>> I’d also like to see some guidelines for what to expect for features > in > >>> the > >>>>> various stages. For example, prior to Production Ready, I would not > >>> expect > >>>>> a feature to be turned on by default for a new broker. > >>>>> > >>>>> The point about barrier to entry is fair, but I think this really > only > >>>>> applies to > >>>>> a handful of large KIPs which take several releases to come to > fruition. > >>>>> I’m sure we didn’t know when KRaft began which release would be the > >>> first > >>>>> in which it would be production ready. I’m sure we decided to release > >>> some > >>>>> early > >>>>> code long before that and having a proper name for that stage seems > >>>>> appropriate. > >>>>> > >>>>> Thanks, > >>>>> Andrew > >>>>> > >>>>>> On 31 Jul 2024, at 18:30, Chris Egerton <fearthecel...@gmail.com> > >>> wrote: > >>>>>> > >>>>>> I'd be hesitant to add too much to the KIP template. We should be > >>> careful > >>>>>> to keep the barrier to entry low for new users. > >>>>>> > >>>>>> On Wed, Jul 31, 2024, 13:24 Kirk True <k...@kirktrue.pro> wrote: > >>>>>> > >>>>>>> Hi Josep, > >>>>>>> > >>>>>>> Thanks for starting this conversation! > >>>>>>> > >>>>>>> Yes, for KIPs that span multiple releases, clearly defining the > KIP’s > >>>>>>> state for each release is very important. Perhaps the KIP template > >>>>> could be > >>>>>>> updated to include a “Release Plan” section that includes entries > that > >>>>>>> include: > >>>>>>> > >>>>>>> - Version: future Apache Kafka release > >>>>>>> - Status: one of Preview, Early access, Production, etc. > >>>>>>> - Jira: umbrella Jira under which other Jiras relevant to that > >>> release > >>>>>>> are kept > >>>>>>> - Notes: Free-form notes as needed by KIP author > >>>>>>> > >>>>>>> Using KIP-848 as an example, the “Release Plan” section could look > >>> like > >>>>>>> this: > >>>>>>> > >>>>>>> - 3.7 > >>>>>>> - Status: Early access > >>>>>>> - Jira: KAFKA-123 > >>>>>>> - Notes: includes support for user-assigned partitions but not > >>>>>>> subscriptions, server-side regex not included > >>>>>>> - 3.8 > >>>>>>> - Status: Preview > >>>>>>> - Jira: KAFKA-234 > >>>>>>> - Notes: includes support for subscriptions and lots of bug > fixes > >>>>>>> - 4.0 > >>>>>>> - Status: Production > >>>>>>> - Jira: KAFKA-345 > >>>>>>> - Notes: includes feature parity with “CLASSIC” consumer groups > >>>>>>> > >>>>>>> Of course, the contents of the “Release Plan” section may need to > >>> change > >>>>>>> over time, and it’s important that the KIP author(s) update it > >>>>> accordingly. > >>>>>>> > >>>>>>> Just a thought. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Kirk > >>>>>>> > >>>>>>>> On Jul 31, 2024, at 9:43 AM, > <https://www.google.com/maps/search/On+Jul+31,+2024,+at+9:43%E2%80%AFAM,?entry=gmail&source=g> > Josep Prat <josep.p...@aiven.io.INVALID > >>>> > >>>>>>> wrote: > >>>>>>>> > >>>>>>>> Hi Colin, > >>>>>>>> > >>>>>>>> I'm not attached to any names, I'm happy to change them. > >>>>>>>> But from what I can guess from KIP-848 the order is EA -> Preview > -> > >>> GA > >>>>>>>> Maybe I'm mistaken. > >>>>>>>> > >>>>>>>> This is why this discussion is important so we all agree on the > >>> number > >>>>> of > >>>>>>>> steps, order and names. > >>>>>>>> > >>>>>>>> I know finding a universal solution is maybe impossible but if we > can > >>>>>>> find > >>>>>>>> one that fits most of the feature releases we had and we are > planning > >>>>>>>> currently would be great. > >>>>>>>> > >>>>>>>> Best, > >>>>>>>> > >>>>>>>> ------------------ > >>>>>>>> Josep Prat > >>>>>>>> Open Source Engineering Director, Aiven > >>>>>>>> josep.p...@aiven.io | +491715557497 | aiven.io > >>>>>>>> Aiven Deutschland GmbH > >>>>>>>> Alexanderufer 3-7, 10117 Berlin > >>>>>>>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, > >>>>>>>> Anna Richardson, Kenneth Chen > >>>>>>>> Amtsgericht Charlottenburg, HRB 209739 B > >>>>>>>> > >>>>>>>> On Wed, Jul 31, 2024, 18:31 Colin McCabe <cmcc...@apache.org> > wrote: > >>>>>>>> > >>>>>>>>> Hi Josep, > >>>>>>>>> > >>>>>>>>> Thanks for bringing up this discussion. > >>>>>>>>> > >>>>>>>>> My impression was that "preview" was earlier than early access. > >>>>>>>>> > >>>>>>>>> In my mind, "preview" somewhat implies that the feature could be > >>>>>>>>> incomplete or partial. For example, if someone said their game > was a > >>>>>>>>> "preview" I would expect kind of a demo. > >>>>>>>>> > >>>>>>>>> Early access (EA) means people can "access" it but it's "early". > In > >>>>>>> other > >>>>>>>>> words, use at your own risk. > >>>>>>>>> > >>>>>>>>> What you are calling "production ready," I think we've sometimes > >>>>> called > >>>>>>>>> generally available (GA) in the past. I don't have a strong > >>> preference > >>>>>>> on > >>>>>>>>> terms. > >>>>>>>>> > >>>>>>>>> I think no matter how much standardization we do, there will > always > >>>>> be a > >>>>>>>>> lot of discretion in these terms :) But maybe this helps clarify > how > >>>>>>>>> they've been used in the past? > >>>>>>>>> > >>>>>>>>> best, > >>>>>>>>> Colin > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Wed, Jul 31, 2024, at 02:03, Josep Prat wrote: > >>>>>>>>>> Hi Kafka devs, > >>>>>>>>>> > >>>>>>>>>> Lately we started using "early access", "production ready" and > also > >>>>>>>>>> "preview" to determine the grade of "production readiness" of > the > >>>>>>>>> features > >>>>>>>>>> we deliver to our community. > >>>>>>>>>> However, as far as I know, there is no official definition from > the > >>>>>>>>> Apache > >>>>>>>>>> Kafka side on which are the graduation steps for features and > what > >>>>> type > >>>>>>>>> of > >>>>>>>>>> "guarantees" each of these offer. > >>>>>>>>>> > >>>>>>>>>> I think we should agree on which terms we should use and what > each > >>> of > >>>>>>>>> these > >>>>>>>>>> exactly mean in terms of reliability. So far it seems we have > this > >>>>>>>>>> graduation steps: > >>>>>>>>>> - Early Access: Feature is just complete but not yet fully > polished > >>>>> and > >>>>>>>>>> maybe not used in production in many environments > >>>>>>>>>> - Preview: Feature was early access before and it underwent at > >>> least > >>>>> a > >>>>>>>>>> cycle of improvements and fixes and it's used in some production > >>>>>>>>>> environments maybe > >>>>>>>>>> - Production ready: Feature is officially rele > >>> < > https://www.google.com/maps/search/ady:+Feature+is+officially+rele?entry=gmail&source=g > >ased > >>> and it fulfills > >>>>> the > >>>>>>>>>> expected initial needs > >>>>>>>>>> > >>>>>>>>>> Note that we don't offer any guarantees or SLA/SLO in the > classical > >>>>>>> term. > >>>>>>>>>> > >>>>>>>>>> Is this something we can agree on? What do those terms mean to > you? > >>>>> Do > >>>>>>> we > >>>>>>>>>> need more steps? Or do we need less steps? > >>>>>>>>>> > >>>>>>>>>> Best, > >>>>>>>>>> -- > >>>>>>>>>> [image: Aiven] <https://www.aiven.io/> > >>>>>>>>>> > >>>>>>>>>> *Josep Prat* > >>>>>>>>>> Open Source Engineering Di > >>>>>>> < > >>>>> > >>> > https://www.google.com/maps/search/%0A%3E%3E%3E+Open+Source+Engineering+Di?entry=gmail&source=g > >>>>>> rector, > >>>>>>> *Aiven* > >>>>>>>>>> josep.p...@aiven.io | +491715557497 > >>>>>>>>>> aiven.io <https://www.aiven.io/> | < > >>>>>>>>> https://www.facebook.com/aivencloud> > >>>>>>>>>> <https://www.linkedin.com/company/aiven/> < > >>>>>>>>> https://twitter.com/aiven_io> > >>>>>>>>>> *Aiven Deutschland GmbH* > >>>>>>>>>> Alexanderufer 3-7, 10117 Berlin > >>>>>>>>>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, > >>>>>>>>>> Anna Richardson, Kenneth Chen > >>>>>>>>>> Amtsgericht Charlottenburg, HRB 209739 B > >>>>>>>>> > >>>>>>> > >>>>>>> > >>>>> > >>>>> > >>>> > >>> >
