Hi Josep, Thanks for creating this KIP. It looks like a good proposal. A few comments.
AS1: I don’t think features should be able to progress between the levels in patch releases. Yes, there may be some bug fixes which mean that the usability of a feature has progressed markedly, but given that Kafka releases are so frequent, it doesn’t seem too much of a burden making features wait until the next minor release to progress to the next level. AS2: While I think that the Graduation Steps Log idea is a good addition to these more complicated KIPs, but I don’t like release managers having to visit all of the KIPs to see what the state of play is. Each release has a release plan on the wiki which people update to add in the KIPs. I would prefer the KIP authors to use that as the way to signal to the release manager that their KIPs have attained a particular level. The Graduation Steps Log is more for the KIP reader to find out when the interesting feature they are reading about was actually delivered in a simple way. The KIP author would have to keep these in sync, which is quite a small burden for something that only changes a few times at most in the life of a KIP. AS3: For KIPs which end up with some pieces being completed before others, it would be simple to document the sub-features which were delivered and graduated at different releases without being too prescriptive and inventing sub-KIPs or whatever. AS4: I suggest adding to the KIP some statement about feature versions, configurations and other switches at the various levels. For example, if a Kafka feature such as “transaction.version” is used to enable a KIP, I suppose it would be introduced at level 3 or later, and switched on by default at level 4. The same with unstable APIs - I supposed at levels 1 and 2, if there are new RPCs defined, I need to use `unstable.api.versions.enable` on the broker to enable them. AS5: I wonder whether we need to avoid “features” here because it already has a specific meaning in Kafka (bin/kafka-features.sh and so on). Maybe this KIP should be “Graduation Steps for KIPs”. AS6: Typo: Level 3 says “This level is optional, and a feature must not go through it”. This should probably say “does not need to”. AS7: I suggest “Incubation” for level 1. Thanks, Andrew > On 22 Aug 2024, at 08:40, Josep Prat <josep.p...@aiven.io.INVALID> wrote: > > Hi Matthias, > > Thanks for your feedback! > - Feedback for level 1: > Yes, indeed it is almost superfluous, but I decided to add it for > completeness. A KIP in level 1 might have some PRs that are merged, but > it's not yet functional nor complete. For example, only the interfaces have > been added. > We could also remove level 1, but then KIPs would start without a level > assigned which is implicitly a level in my opinion. > The reason for level 1 in my opinion is to ease the work of the release > manager in understanding if worked KIPs are to be included in the release > notes and in the list of released KIPs. >> This seems to be little bit confusing. If it's not meant to be used > by anybody, should it not be feature flagged and disable/hidden from users > completely? > Yes, this is correct, most of the time they would be behind a feature flag, > or they will be new code paths not reachable yet. Let's take KIP-853 for > example, at the moment of Kafka 3.8.0 release, only the first 6 tasks under > https://issues.apache.org/jira/browse/KAFKA-14094 were completed. The > feature was not usable and not visible for our users. Thus, level 1. > >> In the end it means, a release does just not contain anything about it > (even if there might be code that was merged), and the KIP should not be > mentioned in the release notes at all, and thus we don't need a name as the > feature is not graduated into any (useful) level yet. > You are totally correct, features in level 1 should not be mentioned in the > release notes. Effectively, the release doesn't contain this KIP, even if > technically it contains work on this KIP. > > The way I understand it, dropping this level from the hierarchy will mean > that KIPs start at no level, and at one point in time go to level X. This > effectively means that KIPs start at an implicit level. Maybe calling it > level 0 helps to better represent that this is the default entry point. > > - Feedback for term usable > Yes this is a good point. But given the variety of features we create, I > don't want to be overly prescriptive and then realize that we don't > represent some in there. If we borrow terms from the industry this could be > "Minimum Usable Product"[1]. Usable for a KIP means that it contains at > least the minimum amount of functionality that makes it usable. What usable > means it varies from KIP to KIP. Let's see KIP-853 again, > https://issues.apache.org/jira/browse/KAFKA-14094 originally contained all > the work needed for the KIP. After a while, the KIP drivers (Jose and > Colin) agreed to split the tasks in 2 groups, the tasks now present under > KAFKA-14094 and the newly created > https://issues.apache.org/jira/browse/KAFKA-17241. > I'm open to suggestions on how we can clarify this better, without being > too prescriptive and then finding that developers can't relate to it. > > - Feedback for partial releases > This is a very good point. This KIP mostly equates features to KIPs, which > at some high level of abstraction probably holds true. But you are > definitely right, bigger KIPs that define a new capability contain > different features. These graduation steps could apply to any of the > sub-features of a KIP. > The bigger and more complex these KIPs are, the more we might want to > logically break them down in the KIP itself. Maybe these KIPs are in the > end sub-KIPs in a KIP, and could each of them follow the same graduation > steps defined here. Also if we think about release announcements, what did > we say about these KIPs that only some parts were completed in a release? > From the list you shared KIP-925: Rack aware task assignment in Kafka > Streams > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-925%3A+Rack+aware+task+assignment+in+Kafka+Streams> > seems to be partially released in 3.6 and then fully released in 3.7. > However, I can't find any trace of KIP-925 in the Kafka 3.6 blog > announcement. Would it reflect the reality to say that KIP-925 had > different "behaviours" that needed to be implemented and some made it > completely (level 4) on to Kafka 3.6, while the last one made it (also > level 4) only in Kafka 3.7? > We could potentially state that KIPs that contain multiple features should > state those clearly, and the graduation steps would apply to these instead > of the KIP itself. > > - Feedback for KIPs that are never completed > I guess this depends on each particular case but I would like for us to > reflect on the situation itself. I think this is something we should > address regardless of this KIP in question. If the features are deemed > important for the KIP, we should keep the KIP "open" until this is done. > If we would agree that big KIPs can list a collection of features and those > are the ones graduating, would then this solve this problem? A KIP will > have some of the features marked at level 4 while others would be at level > 1 (or no level depending on outcome of prior point). > > - Feedback for when to reach new levels > I initially thought the same, level changes can only be triggered in minors > and majors. However, I was thinking that if everything needed for the > feature itself could be done in patch releases why couldn't the graduation > step change in patch releases as well. > For example, let's imagine a fictitious KIP that is on level 3. All public > API changes have been completed already and their implementation is there. > The KIP drivers wanted to seek validation in real scenarios that's why the > feature is at level 3 and not 4. Let's keep imagining that they get > multiple satisfactory reports from the community and a couple of reports > that indicate a performance degradation occurs under circumstance X. This > is fixed and released in the next patch version. Should we be able to > graduate this feature to level 4 in this patch version? > To be honest if people have concerns about this, I'm fine by leaving this > possibly corner case out and only allow graduation changes on major and > minors. > > > Let me know what you think > > Best, > > [1]: https://www.sarvika.com/2021/06/17/mvp-mlp-mup/ contains some > explanation and comparison of MVP vs MLP vs MUP. > > On Thu, Aug 22, 2024 at 7:07 AM Matthias J. Sax <mj...@apache.org> wrote: > >> Thanks Josep. Couple of comments/questions: >> >> For Level 1, the KIP says: >> >>> The feature is not yet stable and it is not meant to be used by end >> users. >> >> This seems to be little bit confusing. If it's not meant to be used by >> anybody, should it not be feature flagged and disable/hidden from users >> completely? >> >> Of course, it should not be used in production, but isn't the goal that >> users do try it out and provide early feedback about the feature? >> However, this is described on Level 2. So I am really not sure what >> Level 1 is supposed to be, or why we would need it as an public >> "contract". Later the KIP say >> >>> Implicitly when a KIP is approved and development starts, the feature is >> effectively in "Level 1" >> >> Which really means to me, we should drop this level from the "public >> hierarchy", as it won't add much (any?) value? >> >> The propose name "In Development" indicates the same thing to me. In the >> end it means, a release does just not contain anything about it (even if >> there might be code that was merged), and the KIP should not be >> mentioned in the release notes at all, and thus we don't need a name as >> the feature is not graduated into any (useful) level yet. >> >> >> I also find the term "usable" very fuzzy. Can we add more color to it? >> >> >> Another thought: we did have cases for KS for which a KIP was only >> partially implemented in a release, but the completed parts where fully >> functional, and thus in stage 4. How do we intent to handle this going >> forward? I would find it rather odd to only release something as stage 2 >> just to follow the process... (as a matter of fact, we still have some >> KIPs which never got fully completed but are pending completion for >> years now, and might frankly never get completed) >> >> Cf >> >> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Streams+KIP+Overview >> for examples >> >> >> >>> A feature can be promoted to a "higher" level at any release (including >> patch releases) >> >> Not sure if this makes sense? To me, a new level can only be reached in >> a major or minor release? >> >> >> >> >> -Matthias >> >> On 8/20/24 6:21 AM, Josep Prat wrote: >>> Hi all, >>> >>> I added a new section in the KIP to specify how to change the graduation >>> levels for a feature. >>> >>> Best! >>> >>> On Mon, Aug 19, 2024 at 4:01 PM Josep Prat <josep.p...@aiven.io> wrote: >>> >>>> Hi TengYao and Chia-Ping, >>>> >>>> I updated the KIP page with examples. >>>> >>>> Best, >>>> >>>> On Mon, Aug 19, 2024 at 2:39 PM TengYao Chi <kiting...@gmail.com> >> wrote: >>>> >>>>> Hi Josep >>>>> >>>>> Thanks for the explanation. I see your point. >>>>> It makes sense to keep these levels distinct for larger initiatives >> like >>>>> KIP-853. I agree with your perspective. >>>>> >>>>> Best regards, >>>>> TengYao >>>>> >>>>> Josep Prat <josep.p...@aiven.io.invalid> 於 2024年8月19日 週一 下午6:36寫道： >>>>> >>>>>> Hi TengYao, >>>>>> >>>>>> I get your point. I think smaller features definitely go too quickly >>>>>> through stages to even acknowledge the change. >>>>>> However, I would still think it's necessary to have these 2 levels >>>>>> separated when it comes to bigger feature initiatives. The biggest use >>>>> case >>>>>> I see right now is to signal to the release manager (and the >> community) >>>>> if >>>>>> a feature is usable or not yet usable. I believe the fact to become >>>>> usable >>>>>> for a feature is a big enough step to gain its own entity. >>>>>> >>>>>> >>>>>> Let's take KIP-853 as an example. This KIP was approved and initially >>>>> added >>>>>> to the release plan for Kafka 3.8. At this point the feature would be >> in >>>>>> Level 1. By the time of the feature freeze the feature was still on >>>>> Level >>>>>> 1, so the release manager (who happened to be me) knew that the KIP >>>>> would >>>>>> not make it in this release and would need to be postponed to a future >>>>>> release. After that, development on this feature continued and it was >>>>>> declared to enter level 2 right in time for being in Kafka 3.9. >>>>>> >>>>>> Let me know what you think. >>>>>> >>>>>> Best, >>>>>> >>>>>> On Mon, Aug 19, 2024 at 8:51 AM TengYao Chi <kiting...@gmail.com> >>>>> wrote: >>>>>> >>>>>>> Hello Josep, >>>>>>> I think this KIP is a great addition to the community that we now >>>>> have a >>>>>>> crystal-clear definition for the state of a feature. >>>>>>> >>>>>>> In the current proposal, I think Level 1 is defined as the stage >>>>> where a >>>>>>> feature is "incomplete and unusable", while Level 2 represents a >>>>> feature >>>>>>> that is "usable but potentially incomplete". >>>>>>> The distinction between these two levels might not always be clear, >>>>>>> especially during the transition of a feature from "unusable" to >>>>> "usable >>>>>>> but incomplete". >>>>>>> >>>>>>> IMHO, to simplify the process and reduce confusion for both >> developers >>>>>> and >>>>>>> users, I would suggest merging Level 1 and Level 2 into a single >>>>> unified >>>>>>> level. >>>>>>> This merged level could cover the entire phase from when a feature is >>>>>>> unstable to when it becomes usable but incomplete. >>>>>>> >>>>>>> WYDT? >>>>>>> >>>>>>> Best regards, >>>>>>> TengYao >>>>>>> >>>>>>> Josep Prat <josep.p...@aiven.io.invalid> 於 2024年8月19日 週一 上午2:58寫道： >>>>>>> >>>>>>>> Hi Chia-Ping, >>>>>>>> >>>>>>>> As far as I can tell, Tiered Storage is still at level 3. I think >>>>> the >>>>>>>> intention would be to declare it level 4 in 4.0.0. >>>>>>>> KIP-848 was in level 2 in Kafka 3.7. and it went level 3 in Kafka >>>>> 3.8. >>>>>>>> Level 4 features would be for example MirrorMaker2 for example. As >>>>> far >>>>>>> as I >>>>>>>> understand the Docker image is level 4. >>>>>>>> >>>>>>>> Does that make sense? If so I can update the KIP with those >>>>> examples. >>>>>>>> >>>>>>>> Best, >>>>>>>> >>>>>>>> ------------------ >>>>>>>> Josep Prat >>>>>>>> Open Source Engineering Director, Aiven >>>>>>>> josep.p...@aiven.io | +491715557497 | aiven.io >>>>>>>> Aiven Deutschland GmbH >>>>>>>> Alexanderufer 3-7, 10117 Berlin >>>>>>>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, >>>>>>>> Anna Richardson, Kenneth Chen >>>>>>>> Amtsgericht Charlottenburg, HRB 209739 B >>>>>>>> >>>>>>>> On Sun, Aug 18, 2024, 21:46 Chia-Ping Tsai <chia7...@gmail.com> >>>>> wrote: >>>>>>>> >>>>>>>>> hi Josep >>>>>>>>> >>>>>>>>> Although I didn't join the discussion before, the KIP is >>>>> interesting >>>>>>> and >>>>>>>>> great to me. >>>>>>>>> >>>>>>>>> one small comment: >>>>>>>>> >>>>>>>>> Could you please add existent features as an example to each level >>>>>> for >>>>>>>> the >>>>>>>>> readers who have poor reading (like me) ? For instance, I guess >>>>> the >>>>>> new >>>>>>>>> coordinator is level 3? tiered storage is level 4? >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Chia-Ping >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Josep Prat <josep.p...@aiven.io.invalid> 於 2024年8月19日 週一 >>>>> 上午2:13寫道： >>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> I want to start a discussion for KIP-1081: Graduation Steps for >>>>>>>> Features. >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1081%3A+Graduation+Steps+for+Features >>>>>>>>>> >>>>>>>>>> We already had a bit of a discussion here >>>>>>>>>> >>>>> https://lists.apache.org/thread/5z6rxvs9m0bro5ssmtg8qcgdk40882bz >>>>>> and >>>>>>>>> that >>>>>>>>>> materialized into this KIP. >>>>>>>>>> >>>>>>>>>> I deliberately defined the graduation steps without giving them >>>>> a >>>>>>> name, >>>>>>>>> so >>>>>>>>>> we don't go bike-shedding there. There is a separate section for >>>>>> the >>>>>>>>> names >>>>>>>>>> of each step. Also an alternative set of names. I'd like to get >>>>>> some >>>>>>>>>> feedback on the steps, and also on the names for the steps. >>>>>>>>>> >>>>>>>>>> Looking forward to your opinions, and hopefully only a tiny bit >>>>> of >>>>>>>>>> bike-shedding :) >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> [image: Aiven] <https://www.aiven.io/> >>>>>>>>>> >>>>>>>>>> *Josep Prat* >>>>>>>>>> Open Source Engineering Director, *Aiven* >>>>>>>>>> josep.p...@aiven.io | +491715557497 >>>>>>>>>> aiven.io <https://www.aiven.io/> | < >>>>>>>>> https://www.facebook.com/aivencloud >>>>>>>>>>> >>>>>>>>>> <https://www.linkedin.com/company/aiven/> < >>>>>>>>>> https://twitter.com/aiven_io> >>>>>>>>>> *Aiven Deutschland GmbH* >>>>>>>>>> Alexanderufer 3-7, 10117 Berlin >>>>>>>>>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, >>>>>>>>>> Anna Richardson, Kenneth Chen >>>>>>>>>> Amtsgericht Charlottenburg, HRB 209739 B >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> [image: Aiven] <https://www.aiven.io/> >>>>>> >>>>>> *Josep Prat* >>>>>> Open Source Engineering Director, *Aiven* >>>>>> josep.p...@aiven.io | +491715557497 >>>>>> aiven.io <https://www.aiven.io/> | < >>>>> https://www.facebook.com/aivencloud >>>>>>> >>>>>> <https://www.linkedin.com/company/aiven/> < >>>>>> https://twitter.com/aiven_io> >>>>>> *Aiven Deutschland GmbH* >>>>>> Alexanderufer 3-7, 10117 Berlin >>>>>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, >>>>>> Anna Richardson, Kenneth Chen >>>>>> Amtsgericht Charlottenburg, HRB 209739 B >>>>>> >>>>> >>>> >>>> >>>> -- >>>> [image: Aiven] <https://www.aiven.io/> >>>> >>>> *Josep Prat* >>>> Open Source Engineering Director, *Aiven* >>>> josep.p...@aiven.io | +491715557497 >>>> aiven.io <https://www.aiven.io/> | >>>> <https://www.facebook.com/aivencloud> >>>> <https://www.linkedin.com/company/aiven/> < >> https://twitter.com/aiven_io> >>>> *Aiven Deutschland GmbH* >>>> Alexanderufer 3-7, 10117 Berlin >>>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, >>>> Anna Richardson, Kenneth Chen >>>> Amtsgericht Charlottenburg, HRB 209739 B >>>> >>> >>> >> > > > -- > [image: Aiven] <https://www.aiven.io/> > > *Josep Prat* > Open Source Engineering Director, *Aiven* > josep.p...@aiven.io | +491715557497 > aiven.io <https://www.aiven.io/> | <https://www.facebook.com/aivencloud> > <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> > *Aiven Deutschland GmbH* > Alexanderufer 3-7, 10117 Berlin > Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, > Anna Richardson, Kenneth Chen > Amtsgericht Charlottenburg, HRB 209739 B
