Hi Andrew, thanks for the feedback!
AS1: I'm fine with it. Updated the KIP. AS2: I think both steps are needed. I agree with you that KIP authors should keep the release plan page up-to-date with the status of their KIP. However, as a user of the KIP, having a graduation steps log would help understanding the state of their feature and to which version they would need to update to gain more stability for example. AS3: Yes, I don't intend to add sub-KIPs. But rather structure your big KIP in functionalities and those functionalities are the ones that can graduate independently. AS4: I can see how a feature in level 2 has also a configuration value to enable it. Let me add a new column to the summary table and descriptions of the levels. AS5: Makes sense, updated the KIP AS6: thanks, updated the KIP. AS7: sounds good to me. Let's see if somebody else has opinions about it. As I changed the title, the new URL is: https://cwiki.apache.org/confluence/x/mYn9Eg On Thu, Aug 22, 2024 at 4:36 PM Andrew Schofield <andrew_schofi...@live.com> wrote: > Hi Josep, > Thanks for creating this KIP. It looks like a good proposal. A few > comments. > > AS1: I don’t think features should be able to progress between the levels > in > patch releases. Yes, there may be some bug fixes which mean that > the usability of a feature has progressed markedly, but given that > Kafka releases are so frequent, it doesn’t seem too much of a burden > making features wait until the next minor release to progress to the > next level. > > AS2: While I think that the Graduation Steps Log idea is a good addition to > these more complicated KIPs, but I don’t like release managers having > to visit all of the KIPs to see what the state of play is. Each release has > a release plan on the wiki which people update to add in the KIPs. > I would prefer the KIP authors to use that as the way to signal to the > release manager that their KIPs have attained a particular level. > The Graduation Steps Log is more for the KIP reader to find out when > the interesting feature they are reading about was actually delivered > in a simple way. The KIP author would have to keep these in sync, > which is quite a small burden for something that only changes a few times > at most in the life of a KIP. > > AS3: For KIPs which end up with some pieces being completed before others, > it would be simple to document the sub-features which were delivered > and graduated at different releases without being too prescriptive and > inventing sub-KIPs or whatever. > > AS4: I suggest adding to the KIP some statement about feature versions, > configurations and other switches at the various levels. For example, > if a Kafka feature such as “transaction.version” is used to enable a KIP, > I suppose it would be introduced at level 3 or later, and switched on > by default at level 4. The same with unstable APIs - I supposed at > levels 1 and 2, if there are new RPCs defined, I need to use > `unstable.api.versions.enable` on the broker to enable them. > > AS5: I wonder whether we need to avoid “features” here because it > already has a specific meaning in Kafka (bin/kafka-features.sh and > so on). Maybe this KIP should be “Graduation Steps for KIPs”. > > AS6: Typo: Level 3 says “This level is optional, and a feature must not go > through it”. This should probably say “does not need to”. > > AS7: I suggest “Incubation” for level 1. > > Thanks, > Andrew > > > > On 22 Aug 2024, at 08:40, Josep Prat <josep.p...@aiven.io.INVALID> > wrote: > > > > Hi Matthias, > > > > Thanks for your feedback! > > - Feedback for level 1: > > Yes, indeed it is almost superfluous, but I decided to add it for > > completeness. A KIP in level 1 might have some PRs that are merged, but > > it's not yet functional nor complete. For example, only the interfaces > have > > been added. > > We could also remove level 1, but then KIPs would start without a level > > assigned which is implicitly a level in my opinion. > > The reason for level 1 in my opinion is to ease the work of the release > > manager in understanding if worked KIPs are to be included in the release > > notes and in the list of released KIPs. > >> This seems to be little bit confusing. If it's not meant to be used > > by anybody, should it not be feature flagged and disable/hidden from > users > > completely? > > Yes, this is correct, most of the time they would be behind a feature > flag, > > or they will be new code paths not reachable yet. Let's take KIP-853 for > > example, at the moment of Kafka 3.8.0 release, only the first 6 tasks > under > > https://issues.apache.org/jira/browse/KAFKA-14094 were completed. The > > feature was not usable and not visible for our users. Thus, level 1. > > > >> In the end it means, a release does just not contain anything about it > > (even if there might be code that was merged), and the KIP should not be > > mentioned in the release notes at all, and thus we don't need a name as > the > > feature is not graduated into any (useful) level yet. > > You are totally correct, features in level 1 should not be mentioned in > the > > release notes. Effectively, the release doesn't contain this KIP, even if > > technically it contains work on this KIP. > > > > The way I understand it, dropping this level from the hierarchy will mean > > that KIPs start at no level, and at one point in time go to level X. This > > effectively means that KIPs start at an implicit level. Maybe calling it > > level 0 helps to better represent that this is the default entry point. > > > > - Feedback for term usable > > Yes this is a good point. But given the variety of features we create, I > > don't want to be overly prescriptive and then realize that we don't > > represent some in there. If we borrow terms from the industry this could > be > > "Minimum Usable Product"[1]. Usable for a KIP means that it contains at > > least the minimum amount of functionality that makes it usable. What > usable > > means it varies from KIP to KIP. Let's see KIP-853 again, > > https://issues.apache.org/jira/browse/KAFKA-14094 originally contained > all > > the work needed for the KIP. After a while, the KIP drivers (Jose and > > Colin) agreed to split the tasks in 2 groups, the tasks now present under > > KAFKA-14094 and the newly created > > https://issues.apache.org/jira/browse/KAFKA-17241. > > I'm open to suggestions on how we can clarify this better, without being > > too prescriptive and then finding that developers can't relate to it. > > > > - Feedback for partial releases > > This is a very good point. This KIP mostly equates features to KIPs, > which > > at some high level of abstraction probably holds true. But you are > > definitely right, bigger KIPs that define a new capability contain > > different features. These graduation steps could apply to any of the > > sub-features of a KIP. > > The bigger and more complex these KIPs are, the more we might want to > > logically break them down in the KIP itself. Maybe these KIPs are in the > > end sub-KIPs in a KIP, and could each of them follow the same graduation > > steps defined here. Also if we think about release announcements, what > did > > we say about these KIPs that only some parts were completed in a release? > > From the list you shared KIP-925: Rack aware task assignment in Kafka > > Streams > > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-925%3A+Rack+aware+task+assignment+in+Kafka+Streams > > > > seems to be partially released in 3.6 and then fully released in 3.7. > > However, I can't find any trace of KIP-925 in the Kafka 3.6 blog > > announcement. Would it reflect the reality to say that KIP-925 had > > different "behaviours" that needed to be implemented and some made it > > completely (level 4) on to Kafka 3.6, while the last one made it (also > > level 4) only in Kafka 3.7? > > We could potentially state that KIPs that contain multiple features > should > > state those clearly, and the graduation steps would apply to these > instead > > of the KIP itself. > > > > - Feedback for KIPs that are never completed > > I guess this depends on each particular case but I would like for us to > > reflect on the situation itself. I think this is something we should > > address regardless of this KIP in question. If the features are deemed > > important for the KIP, we should keep the KIP "open" until this is done. > > If we would agree that big KIPs can list a collection of features and > those > > are the ones graduating, would then this solve this problem? A KIP will > > have some of the features marked at level 4 while others would be at > level > > 1 (or no level depending on outcome of prior point). > > > > - Feedback for when to reach new levels > > I initially thought the same, level changes can only be triggered in > minors > > and majors. However, I was thinking that if everything needed for the > > feature itself could be done in patch releases why couldn't the > graduation > > step change in patch releases as well. > > For example, let's imagine a fictitious KIP that is on level 3. All > public > > API changes have been completed already and their implementation is > there. > > The KIP drivers wanted to seek validation in real scenarios that's why > the > > feature is at level 3 and not 4. Let's keep imagining that they get > > multiple satisfactory reports from the community and a couple of reports > > that indicate a performance degradation occurs under circumstance X. This > > is fixed and released in the next patch version. Should we be able to > > graduate this feature to level 4 in this patch version? > > To be honest if people have concerns about this, I'm fine by leaving this > > possibly corner case out and only allow graduation changes on major and > > minors. > > > > > > Let me know what you think > > > > Best, > > > > [1]: https://www.sarvika.com/2021/06/17/mvp-mlp-mup/ contains some > > explanation and comparison of MVP vs MLP vs MUP. > > > > On Thu, Aug 22, 2024 at 7:07 AM Matthias J. Sax <mj...@apache.org> > wrote: > > > >> Thanks Josep. Couple of comments/questions: > >> > >> For Level 1, the KIP says: > >> > >>> The feature is not yet stable and it is not meant to be used by end > >> users. > >> > >> This seems to be little bit confusing. If it's not meant to be used by > >> anybody, should it not be feature flagged and disable/hidden from users > >> completely? > >> > >> Of course, it should not be used in production, but isn't the goal that > >> users do try it out and provide early feedback about the feature? > >> However, this is described on Level 2. So I am really not sure what > >> Level 1 is supposed to be, or why we would need it as an public > >> "contract". Later the KIP say > >> > >>> Implicitly when a KIP is approved and development starts, the feature > is > >> effectively in "Level 1" > >> > >> Which really means to me, we should drop this level from the "public > >> hierarchy", as it won't add much (any?) value? > >> > >> The propose name "In Development" indicates the same thing to me. In the > >> end it means, a release does just not contain anything about it (even if > >> there might be code that was merged), and the KIP should not be > >> mentioned in the release notes at all, and thus we don't need a name as > >> the feature is not graduated into any (useful) level yet. > >> > >> > >> I also find the term "usable" very fuzzy. Can we add more color to it? > >> > >> > >> Another thought: we did have cases for KS for which a KIP was only > >> partially implemented in a release, but the completed parts where fully > >> functional, and thus in stage 4. How do we intent to handle this going > >> forward? I would find it rather odd to only release something as stage 2 > >> just to follow the process... (as a matter of fact, we still have some > >> KIPs which never got fully completed but are pending completion for > >> years now, and might frankly never get completed) > >> > >> Cf > >> > >> > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Streams+KIP+Overview > >> for examples > >> > >> > >> > >>> A feature can be promoted to a "higher" level at any release (including > >> patch releases) > >> > >> Not sure if this makes sense? To me, a new level can only be reached in > >> a major or minor release? > >> > >> > >> > >> > >> -Matthias > >> > >> On 8/20/24 6:21 AM, Josep Prat wrote: > >>> Hi all, > >>> > >>> I added a new section in the KIP to specify how to change the > graduation > >>> levels for a feature. > >>> > >>> Best! > >>> > >>> On Mon, Aug 19, 2024 at 4:01 PM Josep Prat <josep.p...@aiven.io> > wrote: > >>> > >>>> Hi TengYao and Chia-Ping, > >>>> > >>>> I updated the KIP page with examples. > >>>> > >>>> Best, > >>>> > >>>> On Mon, Aug 19, 2024 at 2:39 PM TengYao Chi <kiting...@gmail.com> > >> wrote: > >>>> > >>>>> Hi Josep > >>>>> > >>>>> Thanks for the explanation. I see your point. > >>>>> It makes sense to keep these levels distinct for larger initiatives > >> like > >>>>> KIP-853. I agree with your perspective. > >>>>> > >>>>> Best regards, > >>>>> TengYao > >>>>> > >>>>> Josep Prat <josep.p...@aiven.io.invalid> 於 2024年8月19日 週一 下午6:36寫道： > >>>>> > >>>>>> Hi TengYao, > >>>>>> > >>>>>> I get your point. I think smaller features definitely go too quickly > >>>>>> through stages to even acknowledge the change. > >>>>>> However, I would still think it's necessary to have these 2 levels > >>>>>> separated when it comes to bigger feature initiatives. The biggest > use > >>>>> case > >>>>>> I see right now is to signal to the release manager (and the > >> community) > >>>>> if > >>>>>> a feature is usable or not yet usable. I believe the fact to become > >>>>> usable > >>>>>> for a feature is a big enough step to gain its own entity. > >>>>>> > >>>>>> > >>>>>> Let's take KIP-853 as an example. This KIP was approved and > initially > >>>>> added > >>>>>> to the release plan for Kafka 3.8. At this point the feature would > be > >> in > >>>>>> Level 1. By the time of the feature freeze the feature was still on > >>>>> Level > >>>>>> 1, so the release manager (who happened to be me) knew that the KIP > >>>>> would > >>>>>> not make it in this release and would need to be postponed to a > future > >>>>>> release. After that, development on this feature continued and it > was > >>>>>> declared to enter level 2 right in time for being in Kafka 3.9. > >>>>>> > >>>>>> Let me know what you think. > >>>>>> > >>>>>> Best, > >>>>>> > >>>>>> On Mon, Aug 19, 2024 at 8:51 AM TengYao Chi <kiting...@gmail.com> > >>>>> wrote: > >>>>>> > >>>>>>> Hello Josep, > >>>>>>> I think this KIP is a great addition to the community that we now > >>>>> have a > >>>>>>> crystal-clear definition for the state of a feature. > >>>>>>> > >>>>>>> In the current proposal, I think Level 1 is defined as the stage > >>>>> where a > >>>>>>> feature is "incomplete and unusable", while Level 2 represents a > >>>>> feature > >>>>>>> that is "usable but potentially incomplete". > >>>>>>> The distinction between these two levels might not always be clear, > >>>>>>> especially during the transition of a feature from "unusable" to > >>>>> "usable > >>>>>>> but incomplete". > >>>>>>> > >>>>>>> IMHO, to simplify the process and reduce confusion for both > >> developers > >>>>>> and > >>>>>>> users, I would suggest merging Level 1 and Level 2 into a single > >>>>> unified > >>>>>>> level. > >>>>>>> This merged level could cover the entire phase from when a feature > is > >>>>>>> unstable to when it becomes usable but incomplete. > >>>>>>> > >>>>>>> WYDT? > >>>>>>> > >>>>>>> Best regards, > >>>>>>> TengYao > >>>>>>> > >>>>>>> Josep Prat <josep.p...@aiven.io.invalid> 於 2024年8月19日 週一 上午2:58寫道： > >>>>>>> > >>>>>>>> Hi Chia-Ping, > >>>>>>>> > >>>>>>>> As far as I can tell, Tiered Storage is still at level 3. I think > >>>>> the > >>>>>>>> intention would be to declare it level 4 in 4.0.0. > >>>>>>>> KIP-848 was in level 2 in Kafka 3.7. and it went level 3 in Kafka > >>>>> 3.8. > >>>>>>>> Level 4 features would be for example MirrorMaker2 for example. As > >>>>> far > >>>>>>> as I > >>>>>>>> understand the Docker image is level 4. > >>>>>>>> > >>>>>>>> Does that make sense? If so I can update the KIP with those > >>>>> examples. > >>>>>>>> > >>>>>>>> Best, > >>>>>>>> > >>>>>>>> ------------------ > >>>>>>>> Josep Prat > >>>>>>>> Open Source Engineering Director, Aiven > >>>>>>>> josep.p...@aiven.io | +491715557497 | aiven.io > >>>>>>>> Aiven Deutschland GmbH > >>>>>>>> Alexanderufer 3-7, 10117 Berlin > >>>>>>>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, > >>>>>>>> Anna Richardson, Kenneth Chen > >>>>>>>> Amtsgericht Charlottenburg, HRB 209739 B > >>>>>>>> > >>>>>>>> On Sun, Aug 18, 2024, 21:46 Chia-Ping Tsai <chia7...@gmail.com> > >>>>> wrote: > >>>>>>>> > >>>>>>>>> hi Josep > >>>>>>>>> > >>>>>>>>> Although I didn't join the discussion before, the KIP is > >>>>> interesting > >>>>>>> and > >>>>>>>>> great to me. > >>>>>>>>> > >>>>>>>>> one small comment: > >>>>>>>>> > >>>>>>>>> Could you please add existent features as an example to each > level > >>>>>> for > >>>>>>>> the > >>>>>>>>> readers who have poor reading (like me) ? For instance, I guess > >>>>> the > >>>>>> new > >>>>>>>>> coordinator is level 3? tiered storage is level 4? > >>>>>>>>> > >>>>>>>>> Best, > >>>>>>>>> Chia-Ping > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Josep Prat <josep.p...@aiven.io.invalid> 於 2024年8月19日 週一 > >>>>> 上午2:13寫道： > >>>>>>>>> > >>>>>>>>>> Hi all, > >>>>>>>>>> I want to start a discussion for KIP-1081: Graduation Steps for > >>>>>>>> Features. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1081%3A+Graduation+Steps+for+Features > >>>>>>>>>> > >>>>>>>>>> We already had a bit of a discussion here > >>>>>>>>>> > >>>>> https://lists.apache.org/thread/5z6rxvs9m0bro5ssmtg8qcgdk40882bz > >>>>>> and > >>>>>>>>> that > >>>>>>>>>> materialized into this KIP. > >>>>>>>>>> > >>>>>>>>>> I deliberately defined the graduation steps without giving them > >>>>> a > >>>>>>> name, > >>>>>>>>> so > >>>>>>>>>> we don't go bike-shedding there. There is a separate section for > >>>>>> the > >>>>>>>>> names > >>>>>>>>>> of each step. Also an alternative set of names. I'd like to get > >>>>>> some > >>>>>>>>>> feedback on the steps, and also on the names for the steps. > >>>>>>>>>> > >>>>>>>>>> Looking forward to your opinions, and hopefully only a tiny bit > >>>>> of > >>>>>>>>>> bike-shedding :) > >>>>>>>>>> > >>>>>>>>>> Best, > >>>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>> [image: Aiven] <https://www.aiven.io/> > >>>>>>>>>> > >>>>>>>>>> *Josep Prat* > >>>>>>>>>> Open Source Engineering Director, *Aiven* > >>>>>>>>>> josep.p...@aiven.io | +491715557497 > >>>>>>>>>> aiven.io <https://www.aiven.io/> | < > >>>>>>>>> https://www.facebook.com/aivencloud > >>>>>>>>>>> > >>>>>>>>>> <https://www.linkedin.com/company/aiven/> < > >>>>>>>>>> https://twitter.com/aiven_io> > >>>>>>>>>> *Aiven Deutschland GmbH* > >>>>>>>>>> Alexanderufer 3-7, 10117 Berlin > >>>>>>>>>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, > >>>>>>>>>> Anna Richardson, Kenneth Chen > >>>>>>>>>> Amtsgericht Charlottenburg, HRB 209739 B > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> [image: Aiven] <https://www.aiven.io/> > >>>>>> > >>>>>> *Josep Prat* > >>>>>> Open Source Engineering Director, *Aiven* > >>>>>> josep.p...@aiven.io | +491715557497 > >>>>>> aiven.io <https://www.aiven.io/> | < > >>>>> https://www.facebook.com/aivencloud > >>>>>>> > >>>>>> <https://www.linkedin.com/company/aiven/> < > >>>>>> https://twitter.com/aiven_io> > >>>>>> *Aiven Deutschland GmbH* > >>>>>> Alexanderufer 3-7, 10117 Berlin > >>>>>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, > >>>>>> Anna Richardson, Kenneth Chen > >>>>>> Amtsgericht Charlottenburg, HRB 209739 B > >>>>>> > >>>>> > >>>> > >>>> > >>>> -- > >>>> [image: Aiven] <https://www.aiven.io/> > >>>> > >>>> *Josep Prat* > >>>> Open Source Engineering Director, *Aiven* > >>>> josep.p...@aiven.io | +491715557497 > >>>> aiven.io <https://www.aiven.io/> | > >>>> <https://www.facebook.com/aivencloud> > >>>> <https://www.linkedin.com/company/aiven/> < > >> https://twitter.com/aiven_io> > >>>> *Aiven Deutschland GmbH* > >>>> Alexanderufer 3-7, 10117 Berlin > >>>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, > >>>> Anna Richardson, Kenneth Chen > >>>> Amtsgericht Charlottenburg, HRB 209739 B > >>>> > >>> > >>> > >> > > > > > > -- > > [image: Aiven] <https://www.aiven.io/> > > > > *Josep Prat* > > Open Source Engineering Director, *Aiven* > > josep.p...@aiven.io | +491715557497 > > aiven.io <https://www.aiven.io/> | < > https://www.facebook.com/aivencloud> > > <https://www.linkedin.com/company/aiven/> < > https://twitter.com/aiven_io> > > *Aiven Deutschland GmbH* > > Alexanderufer 3-7, 10117 Berlin > > Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, > > Anna Richardson, Kenneth Chen > > Amtsgericht Charlottenburg, HRB 209739 B > > -- [image: Aiven] <https://www.aiven.io> *Josep Prat* Open Source Engineering Director, *Aiven* josep.p...@aiven.io | +491715557497 aiven.io <https://www.aiven.io> | <https://www.facebook.com/aivencloud> <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> *Aiven Deutschland GmbH* Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, Anna Richardson, Kenneth Chen Amtsgericht Charlottenburg, HRB 209739 B
