Hi Colin,
>I don't think it makes sense to require that "the feature is usable" at
level 2. As I understand it, this level just means that the feature is
under devlopment. Most features are not usable on day 1 of development.
Similarly, documentation is usually the thing that gets written last. It is
not reasonable to expect it to be written during development, when the
feature might be changing from week to week.
As per the KIP, in "Early Access" we already have something that is usable.
At that point I think it can be reasonable to expect something like "here
is how you turn it on". In most cases where we have a feature flag, it
already has this minimal level of documentation, since with configs we
require a short description. I don't expect here that features should talk
about a lot of stuff because as you said, the API might be unstable so we
may not have much else to document. In "Preview" mode however, where we
expect the API to be stable at least, we could expect that some of the
parts (at least the APIs) can be documented and the rest can be added in
"Production Ready". Overall I think it's useful if we make documentation
when we can say something is stable because later people may not remember
as well or it's too big of a burden to write extensive docs.
>I think we should avoid turning this into a code quality checklist. It
really should be about when the feature is ready to be used by end-users.
And a lot of KIPs don't involve code at all, like deprecating some
configuration, changing the Scala version, changing the JDK version, etc.
>We already added a section about "Testing" to each KIP. Really the
requirement to reach the last level should be that you did all the testing
that you promised to do. If that testing was insufficient, then that
problem should be identified during the KIP discussion.
In my mind the readiness level of features and some notion of quality is a
bit intertwined as "Production Ready" means to me that it's high quality
and well tested. As you mention too, there are cases where new tests are
not required and that is fine, I don't think either that it is a must do
requirement for all features, just where the given KIP states it. I also
didn't want this to be a big checklist. I agree that the KIPs should
specify what levels of testing should be done (which we suppose is enough
if the KIP is approved) and a feature shouldn't be production ready if the
tests promised weren't implemented. I just wanted to make a case for
requiring some set of these tests identified in the KIP to be implemented
in phases to achieve some benefits. In my experience it's good if tests are
developed together gradually with the feature. If stable parts are tested,
then later modifications are less prone to regressions. Having tests
earlier is also good for easing development or involving others in the
development.
We may loosen my requirements to something like
- we don't require tests in "Early Access" as there is nothing stable (but
encourage happy path tests if possible)
- IF unit/integration tests are required, then "Preview" should have the
happy path tested
- IF unit/integration tests are required, then "Production Ready" should
have all the tests that are promised
What do you think?
>Which names are "the alternative names" to you? :)
"In Development"
"Manuscript"
"Pre-print"
"Published"
I think they're unique, not boring and add some spice to the project :).
They are also free from the preconceptions of the other alternatives you
all discussed previously.
Best,
Viktor
On Mon, Aug 26, 2024 at 7:51 PM Josep Prat <josep.p...@aiven.io.invalid>
wrote:
> Hi Colin,
>
> Names are in the KIP. Level 1 to 4 are never meant to be used outside of
> this discussion. It's my, apparently successful, attempt to focus on what
> the levels mean instead of their names:
>
> > Names
>
> "In Development"
> "Early Access"
> "Preview"
> "Production Ready"
>
> Alternatively, if we want to be a bit more playful we could go with a theme
> borrowed from the book industry (as an homage to Franz Kafka):
>
> "In Development"
> "Manuscript"
> "Pre-print"
> "Published"
>
>
> Regarding level 2, it is intended to be usable up to some extent. For
> example all the common flows are implemented but some corner cases might
> not be built yet:
> > Level 2
> >
> > The KIP is now usable, but it might not be completed yet.
>
> So the main difference between Level 1 and 2 is that in level 1 the feature
> can't be used. In level 2 let's say the common flows are. Level 3 all flows
> (common and corner cases) are implemented.
>
> Does that make sense?
>
>
> Best,
>
> ------------------
> Josep Prat
> Open Source Engineering Director, Aiven
> josep.p...@aiven.io | +10491715557497 | aiven.io
> Aiven Deutschland GmbH
> Alexanderufer 3-7, 10117 Berlin
> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
> Anna Richardson, Kenneth Chen
> Amtsgericht Charlottenburg, HRB 209739 B
>
> On Mon, Aug 26, 2024, 19:05 Colin McCabe <cmcc...@apache.org> wrote:
>
> > On Mon, Aug 26, 2024, at 08:17, Viktor Somogyi-Vass wrote:
> > > Hi all,
> > >
> > > A couple of additional questions/suggestions regarding the KIP:
> > >
> > > VS1. Testing: you wrote about testing only on level 4 but I think we
> > should
> > > add some testing criteria to each one to ensure that it meets
> > > those requirements specified by the given level and it is gradually
> more
> > > mature. For instance for level 1 I think we may not need to do any
> > > integration testing. For level 2 I think we should at least have happy
> > path
> > > integration tests where applicable and do full integration testing on
> > level
> > > 3. Of course smaller KIPs may skip level 2 as you mentioned or have
> > > different testing requirements, so it may not be applicable everywhere.
> > >
> > > VS2. Documentation: I think documentation and user resources (tutorial)
> > is
> > > also a bit similar. On level 1 and maybe 2 I wouldn't expect any
> > > documentation but I would be happy with a feature doc on level 3 and
> > level
> > > 4 may also need to provide examples (if applicable). For instance in
> the
> > > case of a KRaft or MM2 level feature, I would only consider it level 4
> > > (among other things) if it had a very extensive documentation and
> > > tutorials/examples for users.
> > >
> > > VS3. Checklist: I guess the bigger point of view of my previous points
> is
> > > that we should create a checklist of a few items (not too many) that
> KIPs
> > > graduating to that level should fulfill (besides the ones on the
> previous
> > > level) to ensure it is really some sort of an objective graduation. In
> my
> > > mind it looks like this:
> > > Level 1:
> > > - the KIP has to be accepted
> > > Level 2:
> > > - the feature is usable
> > > - has integration tests for the happy path
> > > - unit tests exists that cover the existing functionality
> > > - some minimal documentation exists for early adopters
> >
> > Hi Viktor,
> >
> > I don't think it makes sense to require that "the feature is usable" at
> > level 2. As I understand it, this level just means that the feature is
> > under devlopment. Most features are not usable on day 1 of development.
> > Similarly, documentation is usually the thing that gets written last. It
> is
> > not reasonable to expect it to be written during development, when the
> > feature might be changing from week to week.
> >
> > > Level 3:
> > > - stable API
> > > - integration tests cover all paths
> > > - unit tests cover added functionality
> > > - documentation exists for users
> > > Level 4:
> > > - extensive documentation exists for users with examples or tutorials
> > if
> > > needed
> > > - unit tests cover added functionality
> > > - integration test suites covering the KIPs functionality
> > > - system tests if needed
> >
> > I think we should avoid turning this into a code quality checklist. It
> > really should be about when the feature is ready to be used by end-users.
> > And a lot of KIPs don't involve code at all, like deprecating some
> > configuration, changing the Scala version, changing the JDK version, etc.
> >
> > We already added a section about "Testing" to each KIP. Really the
> > requirement to reach the last level should be that you did all the
> testing
> > that you promised to do. If that testing was insufficient, then that
> > problem should be identified during the KIP discussion.
> >
> > >
> > > PS. I like the alternative names :)
> > >
> >
> > Which names are "the alternative names" to you? :)
> >
> > As I said earlier, I'm not happy with the "level N" jargon since I don't
> > think people outside this dev mailing list will understand it. Most users
> > will react to "that feature is on level 2" with incomprehension. On the
> > other hand, if you tell them that the feature is "alpha," they'll get
> what
> > you're saying. Let's not add jargon that our users won't understand.
> >
> > best,
> > Colin
> >
> >
> > > Best,
> > > Viktor
> > >
> > > On Mon, Aug 26, 2024 at 11:20 AM Josep Prat
> <josep.p...@aiven.io.invalid
> > >
> > > wrote:
> > >
> > >> Hi David,
> > >>
> > >> Thanks for the feedback!
> > >>
> > >> DA1. I'm fine not exposing level 1, but I think it was worth having it
> > for
> > >> completeness-sake as you mention. This level is probably more of a
> > >> technicality but my state-machine brain needs the initial state.
> > >>
> > >> DA2. Yes, this is the key difference between level 3 and 4. Not all
> > >> features need to go through level 3, for example, refactoring APIs or
> > >> adding new public methods for convenience can directly go to level 4.
> > So I
> > >> see level 3 as the default "rest" level for "big" features until we
> gain
> > >> certainty. While "simpler" features could go up to level 4 directly.
> > >>
> > >> DA3. This is a good suggestion. I don't know if we can be too
> > prescriptive
> > >> with this. It all would boil down to the amount and quality of
> feedback
> > >> from the early adopters. Now the KIP mentions that levels can only be
> > >> changed in minors and majors, this means that if we don't say anything
> > >> else, the minimum "baking time" would be 1 minor release. This is the
> > >> technical lower limit. We could mention that we encourage to gather
> > >> feedback from the community for 2 minor releases (the one where the
> > feature
> > >> was released at level 3 and the next minor release). So a feature
> > reaching
> > >> level 3 in Kafka 4.0, could technically change to level 4.1, but it is
> > >> encouraged to wait at least until 4.2.
> > >>
> > >> DA4. I see your point. I did some archeology to write this KIP and as
> I
> > >> mention in the motivation, prior 2.8 all features were released as
> what
> > we
> > >> could assume would be production ready. If I'm not mistaken, the
> policy
> > for
> > >> critical bug backporting has always been 3 releases. I wouldn't change
> > this
> > >> in this KIP and leave it for further KIPs to raise this if it turns
> out
> > to
> > >> be necessary.
> > >>
> > >> DA5. We could potentially do the following state lifecycle: "Under
> > >> Discussion", "Voting", "Rejected", "Accepted" (this would equal level
> > 1),
> > >> "level 2", "level 3", level 4". What do you think?
> > >>
> > >> DA6. This was my intention with the "Graduation Steps Log". Each KIP
> > that
> > >> needs in between states, would have this and users could see when each
> > KIP
> > >> transitioned to each level. If you want to have the reverse lookup
> (per
> > >> release). We could add a new section in the "upgrading" documentation
> > >> section. We could include all KIPs with their state in there.
> > >>
> > >> Let me know what you think!
> > >>
> > >> Best!
> > >>
> > >> On Sun, Aug 25, 2024 at 11:50 PM David Arthur <mum...@gmail.com>
> wrote:
> > >>
> > >> > Josep, thanks for the KIP! I think clear definitions of feature
> > stability
> > >> > will be a boon to our users and developers.
> > >> >
> > >> > DA1. I agree with others in this thread that three levels is
> probably
> > >> > sufficient. Anything before "usable" is not really necessary to
> define
> > >> from
> > >> > a user's perspective. It's fine to include this definition in the
> KIP
> > for
> > >> > completeness-sake, but I don't think we should not expose it as part
> > of
> > >> the
> > >> > "feature stability API".
> > >> >
> > >> > DA2. To me, one of the biggest differences between 3 and 4 is
> feedback
> > >> from
> > >> > production experience. We can do our best to test new things, but
> > until
> > >> > something is used "in anger" we will not really be able to say it's
> > ready
> > >> > for production. This is especially true for large features. I would
> > say
> > >> > level 3 is what we (Kafka) normally call a new feature. We can't
> > really
> > >> > promote a feature to level 4 until we (or our users) have gained
> > >> production
> > >> > experience with it. This means we will rely on early adopters for
> > >> feedback
> > >> > regarding new features.
> > >> >
> > >> > DA3. Related to the DA2, Should we consider a number of releases or
> > >> amount
> > >> > of time it takes for a feature to move from Level 3 to 4? Some
> minimum
> > >> > "bake time" essentially
> > >> >
> > >> > DA4. I think we should also talk about bug backports in this KIP.
> > Once we
> > >> > announce something as level 4, what is our bug backport policy?
> > >> > Unofficially, I think we do something like two minor releases. For
> > >> example,
> > >> > if we promoted a feature to level 4 in 4.1, and trunk is now on 4.7
> --
> > >> how
> > >> > far back do we backport critical fixes? Every release since the
> level
> > 4
> > >> > promotion, or only a few?
> > >> >
> > >> > DA5. Should we add new states for our KIPs? Today, we end at
> Adopted.
> > It
> > >> > might make sense to have new states corresponding to the feature
> > >> stability.
> > >> > It would also be good to capture this information in the KIP itself
> > >> under a
> > >> > new "History" section.
> > >> >
> > >> > DA6. How will we convey this information to our users? Right now,
> > >> > announcements of features are scattered across release notes. It
> > would be
> > >> > really nice if we could come up with a way to easily look up the
> state
> > >> of a
> > >> > feature/KIP at a particular version.
> > >> >
> > >> > Cheers,
> > >> > David A
> > >> >
> > >> > On Fri, Aug 23, 2024 at 1:08 PM Colin McCabe <cmcc...@apache.org>
> > wrote:
> > >> >
> > >> > > On Thu, Aug 22, 2024, at 07:36, Andrew Schofield wrote:
> > >> > > > Hi Josep,
> > >> > > > Thanks for creating this KIP. It looks like a good proposal. A
> few
> > >> > > comments.
> > >> > > >
> > >> > > > AS1: I don’t think features should be able to progress between
> the
> > >> > > levels in
> > >> > > > patch releases. Yes, there may be some bug fixes which mean that
> > >> > > > the usability of a feature has progressed markedly, but given
> that
> > >> > > > Kafka releases are so frequent, it doesn’t seem too much of a
> > burden
> > >> > > > making features wait until the next minor release to progress to
> > the
> > >> > > > next level.
> > >> > > >
> > >> > >
> > >> > > Hi Andrew,
> > >> > >
> > >> > > I think that's fine going forward, but we need to clarify that it
> > only
> > >> > > applies to new features started after this KIP. One reason is
> > because
> > >> the
> > >> > > plan for KIP-853 is to make it production-ready in one of the
> point
> > >> > > releases of 3.9.
> > >> > >
> > >> > > > AS2: While I think that the Graduation Steps Log idea is a good
> > >> > addition
> > >> > > to
> > >> > > > these more complicated KIPs, but I don’t like release managers
> > having
> > >> > > > to visit all of the KIPs to see what the state of play is. Each
> > >> release
> > >> > > has
> > >> > > > a release plan on the wiki which people update to add in the
> KIPs.
> > >> > > > I would prefer the KIP authors to use that as the way to signal
> to
> > >> the
> > >> > > > release manager that their KIPs have attained a particular
> level.
> > >> > > > The Graduation Steps Log is more for the KIP reader to find out
> > when
> > >> > > > the interesting feature they are reading about was actually
> > delivered
> > >> > > > in a simple way. The KIP author would have to keep these in
> sync,
> > >> > > > which is quite a small burden for something that only changes a
> > few
> > >> > times
> > >> > > > at most in the life of a KIP.
> > >> > >
> > >> > > I think realistically, the release managers will have to check up
> on
> > >> the
> > >> > > status of KIPs... for example, by pinging the people working on
> > them.
> > >> > > Otherwise we'll have stuff stuck in an older state even after it's
> > >> ready,
> > >> > > because the devs forgot to update it. This is similar to the work
> > >> release
> > >> > > managers already do to update the release version of KIPs. I agree
> > that
> > >> > > it's good to encourage the developers to do this on their own
> > >> initiative
> > >> > as
> > >> > > well.
> > >> > >
> > >> > > Also, for KIPs that don't need multiple steps, I wonder if we need
> > >> > > anything different than what we have now (just a "release version"
> > >> > field).
> > >> > >
> > >> > > best,
> > >> > > Colin
> > >> > >
> > >> > > >
> > >> > > > AS3: For KIPs which end up with some pieces being completed
> before
> > >> > > others,
> > >> > > > it would be simple to document the sub-features which were
> > delivered
> > >> > > > and graduated at different releases without being too
> prescriptive
> > >> and
> > >> > > > inventing sub-KIPs or whatever.
> > >> > > >
> > >> > > > AS4: I suggest adding to the KIP some statement about feature
> > >> versions,
> > >> > > > configurations and other switches at the various levels. For
> > example,
> > >> > > > if a Kafka feature such as “transaction.version” is used to
> > enable a
> > >> > KIP,
> > >> > > > I suppose it would be introduced at level 3 or later, and
> > switched on
> > >> > > > by default at level 4. The same with unstable APIs - I supposed
> at
> > >> > > > levels 1 and 2, if there are new RPCs defined, I need to use
> > >> > > > `unstable.api.versions.enable` on the broker to enable them.
> > >> > > >
> > >> > > > AS5: I wonder whether we need to avoid “features” here because
> it
> > >> > > > already has a specific meaning in Kafka (bin/kafka-features.sh
> and
> > >> > > > so on). Maybe this KIP should be “Graduation Steps for KIPs”.
> > >> > > >
> > >> > > > AS6: Typo: Level 3 says “This level is optional, and a feature
> > must
> > >> not
> > >> > > go
> > >> > > > through it”. This should probably say “does not need to”.
> > >> > > >
> > >> > > > AS7: I suggest “Incubation” for level 1.
> > >> > > >
> > >> > > > Thanks,
> > >> > > > Andrew
> > >> > > >
> > >> > > >
> > >> > > >> On 22 Aug 2024, at 08:40, Josep Prat
> <josep.p...@aiven.io.INVALID
> > >
> > >> > > wrote:
> > >> > > >>
> > >> > > >> Hi Matthias,
> > >> > > >>
> > >> > > >> Thanks for your feedback!
> > >> > > >> - Feedback for level 1:
> > >> > > >> Yes, indeed it is almost superfluous, but I decided to add it
> for
> > >> > > >> completeness. A KIP in level 1 might have some PRs that are
> > merged,
> > >> > but
> > >> > > >> it's not yet functional nor complete. For example, only the
> > >> interfaces
> > >> > > have
> > >> > > >> been added.
> > >> > > >> We could also remove level 1, but then KIPs would start
> without a
> > >> > level
> > >> > > >> assigned which is implicitly a level in my opinion.
> > >> > > >> The reason for level 1 in my opinion is to ease the work of the
> > >> > release
> > >> > > >> manager in understanding if worked KIPs are to be included in
> the
> > >> > > release
> > >> > > >> notes and in the list of released KIPs.
> > >> > > >>> This seems to be little bit confusing. If it's not meant to be
> > used
> > >> > > >> by anybody, should it not be feature flagged and disable/hidden
> > from
> > >> > > users
> > >> > > >> completely?
> > >> > > >> Yes, this is correct, most of the time they would be behind a
> > >> feature
> > >> > > flag,
> > >> > > >> or they will be new code paths not reachable yet. Let's take
> > KIP-853
> > >> > for
> > >> > > >> example, at the moment of Kafka 3.8.0 release, only the first 6
> > >> tasks
> > >> > > under
> > >> > > >> https://issues.apache.org/jira/browse/KAFKA-14094 were
> > completed.
> > >> The
> > >> > > >> feature was not usable and not visible for our users. Thus,
> > level 1.
> > >> > > >>
> > >> > > >>> In the end it means, a release does just not contain anything
> > about
> > >> > it
> > >> > > >> (even if there might be code that was merged), and the KIP
> should
> > >> not
> > >> > be
> > >> > > >> mentioned in the release notes at all, and thus we don't need a
> > name
> > >> > as
> > >> > > the
> > >> > > >> feature is not graduated into any (useful) level yet.
> > >> > > >> You are totally correct, features in level 1 should not be
> > mentioned
> > >> > in
> > >> > > the
> > >> > > >> release notes. Effectively, the release doesn't contain this
> KIP,
> > >> even
> > >> > > if
> > >> > > >> technically it contains work on this KIP.
> > >> > > >>
> > >> > > >> The way I understand it, dropping this level from the hierarchy
> > will
> > >> > > mean
> > >> > > >> that KIPs start at no level, and at one point in time go to
> > level X.
> > >> > > This
> > >> > > >> effectively means that KIPs start at an implicit level. Maybe
> > >> calling
> > >> > it
> > >> > > >> level 0 helps to better represent that this is the default
> entry
> > >> > point.
> > >> > > >>
> > >> > > >> - Feedback for term usable
> > >> > > >> Yes this is a good point. But given the variety of features we
> > >> > create, I
> > >> > > >> don't want to be overly prescriptive and then realize that we
> > don't
> > >> > > >> represent some in there. If we borrow terms from the industry
> > this
> > >> > > could be
> > >> > > >> "Minimum Usable Product"[1]. Usable for a KIP means that it
> > contains
> > >> > at
> > >> > > >> least the minimum amount of functionality that makes it usable.
> > What
> > >> > > usable
> > >> > > >> means it varies from KIP to KIP. Let's see KIP-853 again,
> > >> > > >> https://issues.apache.org/jira/browse/KAFKA-14094 originally
> > >> > contained
> > >> > > all
> > >> > > >> the work needed for the KIP. After a while, the KIP drivers
> (Jose
> > >> and
> > >> > > >> Colin) agreed to split the tasks in 2 groups, the tasks now
> > present
> > >> > > under
> > >> > > >> KAFKA-14094 and the newly created
> > >> > > >> https://issues.apache.org/jira/browse/KAFKA-17241.
> > >> > > >> I'm open to suggestions on how we can clarify this better,
> > without
> > >> > being
> > >> > > >> too prescriptive and then finding that developers can't relate
> to
> > >> it.
> > >> > > >>
> > >> > > >> - Feedback for partial releases
> > >> > > >> This is a very good point. This KIP mostly equates features to
> > KIPs,
> > >> > > which
> > >> > > >> at some high level of abstraction probably holds true. But you
> > are
> > >> > > >> definitely right, bigger KIPs that define a new capability
> > contain
> > >> > > >> different features. These graduation steps could apply to any
> of
> > the
> > >> > > >> sub-features of a KIP.
> > >> > > >> The bigger and more complex these KIPs are, the more we might
> > want
> > >> to
> > >> > > >> logically break them down in the KIP itself. Maybe these KIPs
> > are in
> > >> > the
> > >> > > >> end sub-KIPs in a KIP, and could each of them follow the same
> > >> > graduation
> > >> > > >> steps defined here. Also if we think about release
> announcements,
> > >> what
> > >> > > did
> > >> > > >> we say about these KIPs that only some parts were completed in
> a
> > >> > > release?
> > >> > > >> From the list you shared KIP-925: Rack aware task assignment in
> > >> Kafka
> > >> > > >> Streams
> > >> > > >> <
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-925%3A+Rack+aware+task+assignment+in+Kafka+Streams
> > >> > > >
> > >> > > >> seems to be partially released in 3.6 and then fully released
> in
> > >> 3.7.
> > >> > > >> However, I can't find any trace of KIP-925 in the Kafka 3.6
> blog
> > >> > > >> announcement. Would it reflect the reality to say that KIP-925
> > had
> > >> > > >> different "behaviours" that needed to be implemented and some
> > made
> > >> it
> > >> > > >> completely (level 4) on to Kafka 3.6, while the last one made
> it
> > >> (also
> > >> > > >> level 4) only in Kafka 3.7?
> > >> > > >> We could potentially state that KIPs that contain multiple
> > features
> > >> > > should
> > >> > > >> state those clearly, and the graduation steps would apply to
> > these
> > >> > > instead
> > >> > > >> of the KIP itself.
> > >> > > >>
> > >> > > >> - Feedback for KIPs that are never completed
> > >> > > >> I guess this depends on each particular case but I would like
> > for us
> > >> > to
> > >> > > >> reflect on the situation itself. I think this is something we
> > should
> > >> > > >> address regardless of this KIP in question. If the features are
> > >> deemed
> > >> > > >> important for the KIP, we should keep the KIP "open" until this
> > is
> > >> > done.
> > >> > > >> If we would agree that big KIPs can list a collection of
> features
> > >> and
> > >> > > those
> > >> > > >> are the ones graduating, would then this solve this problem? A
> > KIP
> > >> > will
> > >> > > >> have some of the features marked at level 4 while others would
> > be at
> > >> > > level
> > >> > > >> 1 (or no level depending on outcome of prior point).
> > >> > > >>
> > >> > > >> - Feedback for when to reach new levels
> > >> > > >> I initially thought the same, level changes can only be
> > triggered in
> > >> > > minors
> > >> > > >> and majors. However, I was thinking that if everything needed
> for
> > >> the
> > >> > > >> feature itself could be done in patch releases why couldn't the
> > >> > > graduation
> > >> > > >> step change in patch releases as well.
> > >> > > >> For example, let's imagine a fictitious KIP that is on level 3.
> > All
> > >> > > public
> > >> > > >> API changes have been completed already and their
> implementation
> > is
> > >> > > there.
> > >> > > >> The KIP drivers wanted to seek validation in real scenarios
> > that's
> > >> why
> > >> > > the
> > >> > > >> feature is at level 3 and not 4. Let's keep imagining that they
> > get
> > >> > > >> multiple satisfactory reports from the community and a couple
> of
> > >> > reports
> > >> > > >> that indicate a performance degradation occurs under
> > circumstance X.
> > >> > > This
> > >> > > >> is fixed and released in the next patch version. Should we be
> > able
> > >> to
> > >> > > >> graduate this feature to level 4 in this patch version?
> > >> > > >> To be honest if people have concerns about this, I'm fine by
> > leaving
> > >> > > this
> > >> > > >> possibly corner case out and only allow graduation changes on
> > major
> > >> > and
> > >> > > >> minors.
> > >> > > >>
> > >> > > >>
> > >> > > >> Let me know what you think
> > >> > > >>
> > >> > > >> Best,
> > >> > > >>
> > >> > > >> [1]: https://www.sarvika.com/2021/06/17/mvp-mlp-mup/ contains
> > some
> > >> > > >> explanation and comparison of MVP vs MLP vs MUP.
> > >> > > >>
> > >> > > >> On Thu, Aug 22, 2024 at 7:07 AM Matthias J. Sax <
> > mj...@apache.org>
> > >> > > wrote:
> > >> > > >>
> > >> > > >>> Thanks Josep. Couple of comments/questions:
> > >> > > >>>
> > >> > > >>> For Level 1, the KIP says:
> > >> > > >>>
> > >> > > >>>> The feature is not yet stable and it is not meant to be used
> by
> > >> end
> > >> > > >>> users.
> > >> > > >>>
> > >> > > >>> This seems to be little bit confusing. If it's not meant to be
> > used
> > >> > by
> > >> > > >>> anybody, should it not be feature flagged and disable/hidden
> > from
> > >> > users
> > >> > > >>> completely?
> > >> > > >>>
> > >> > > >>> Of course, it should not be used in production, but isn't the
> > goal
> > >> > that
> > >> > > >>> users do try it out and provide early feedback about the
> > feature?
> > >> > > >>> However, this is described on Level 2. So I am really not sure
> > what
> > >> > > >>> Level 1 is supposed to be, or why we would need it as an
> public
> > >> > > >>> "contract". Later the KIP say
> > >> > > >>>
> > >> > > >>>> Implicitly when a KIP is approved and development starts, the
> > >> > feature
> > >> > > is
> > >> > > >>> effectively in "Level 1"
> > >> > > >>>
> > >> > > >>> Which really means to me, we should drop this level from the
> > >> "public
> > >> > > >>> hierarchy", as it won't add much (any?) value?
> > >> > > >>>
> > >> > > >>> The propose name "In Development" indicates the same thing to
> > me.
> > >> In
> > >> > > the
> > >> > > >>> end it means, a release does just not contain anything about
> it
> > >> (even
> > >> > > if
> > >> > > >>> there might be code that was merged), and the KIP should not
> be
> > >> > > >>> mentioned in the release notes at all, and thus we don't need
> a
> > >> name
> > >> > as
> > >> > > >>> the feature is not graduated into any (useful) level yet.
> > >> > > >>>
> > >> > > >>>
> > >> > > >>> I also find the term "usable" very fuzzy. Can we add more
> color
> > to
> > >> > it?
> > >> > > >>>
> > >> > > >>>
> > >> > > >>> Another thought: we did have cases for KS for which a KIP was
> > only
> > >> > > >>> partially implemented in a release, but the completed parts
> > where
> > >> > fully
> > >> > > >>> functional, and thus in stage 4. How do we intent to handle
> this
> > >> > going
> > >> > > >>> forward? I would find it rather odd to only release something
> as
> > >> > stage
> > >> > > 2
> > >> > > >>> just to follow the process... (as a matter of fact, we still
> > have
> > >> > some
> > >> > > >>> KIPs which never got fully completed but are pending
> completion
> > for
> > >> > > >>> years now, and might frankly never get completed)
> > >> > > >>>
> > >> > > >>> Cf
> > >> > > >>>
> > >> > > >>>
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Streams+KIP+Overview
> > >> > > >>> for examples
> > >> > > >>>
> > >> > > >>>
> > >> > > >>>
> > >> > > >>>> A feature can be promoted to a "higher" level at any release
> > >> > > (including
> > >> > > >>> patch releases)
> > >> > > >>>
> > >> > > >>> Not sure if this makes sense? To me, a new level can only be
> > >> reached
> > >> > in
> > >> > > >>> a major or minor release?
> > >> > > >>>
> > >> > > >>>
> > >> > > >>>
> > >> > > >>>
> > >> > > >>> -Matthias
> > >> > > >>>
> > >> > > >>> On 8/20/24 6:21 AM, Josep Prat wrote:
> > >> > > >>>> Hi all,
> > >> > > >>>>
> > >> > > >>>> I added a new section in the KIP to specify how to change the
> > >> > > graduation
> > >> > > >>>> levels for a feature.
> > >> > > >>>>
> > >> > > >>>> Best!
> > >> > > >>>>
> > >> > > >>>> On Mon, Aug 19, 2024 at 4:01 PM Josep Prat <
> > josep.p...@aiven.io>
> > >> > > wrote:
> > >> > > >>>>
> > >> > > >>>>> Hi TengYao and Chia-Ping,
> > >> > > >>>>>
> > >> > > >>>>> I updated the KIP page with examples.
> > >> > > >>>>>
> > >> > > >>>>> Best,
> > >> > > >>>>>
> > >> > > >>>>> On Mon, Aug 19, 2024 at 2:39 PM TengYao Chi <
> > kiting...@gmail.com
> > >> >
> > >> > > >>> wrote:
> > >> > > >>>>>
> > >> > > >>>>>> Hi Josep
> > >> > > >>>>>>
> > >> > > >>>>>> Thanks for the explanation. I see your point.
> > >> > > >>>>>> It makes sense to keep these levels distinct for larger
> > >> > initiatives
> > >> > > >>> like
> > >> > > >>>>>> KIP-853. I agree with your perspective.
> > >> > > >>>>>>
> > >> > > >>>>>> Best regards,
> > >> > > >>>>>> TengYao
> > >> > > >>>>>>
> > >> > > >>>>>> Josep Prat <josep.p...@aiven.io.invalid> 於 2024年8月19日 週一
> > >> > 下午6:36寫道：
> > >> > > >>>>>>
> > >> > > >>>>>>> Hi TengYao,
> > >> > > >>>>>>>
> > >> > > >>>>>>> I get your point. I think smaller features definitely go
> too
> > >> > > quickly
> > >> > > >>>>>>> through stages to even acknowledge the change.
> > >> > > >>>>>>> However, I would still think it's necessary to have these
> 2
> > >> > levels
> > >> > > >>>>>>> separated when it comes to bigger feature initiatives. The
> > >> > biggest
> > >> > > use
> > >> > > >>>>>> case
> > >> > > >>>>>>> I see right now is to signal to the release manager (and
> the
> > >> > > >>> community)
> > >> > > >>>>>> if
> > >> > > >>>>>>> a feature is usable or not yet usable. I believe the fact
> to
> > >> > become
> > >> > > >>>>>> usable
> > >> > > >>>>>>> for a feature is a big enough step to gain its own entity.
> > >> > > >>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>> Let's take KIP-853 as an example. This KIP was approved
> and
> > >> > > initially
> > >> > > >>>>>> added
> > >> > > >>>>>>> to the release plan for Kafka 3.8. At this point the
> feature
> > >> > would
> > >> > > be
> > >> > > >>> in
> > >> > > >>>>>>> Level 1. By the time of the feature freeze the feature was
> > >> still
> > >> > on
> > >> > > >>>>>> Level
> > >> > > >>>>>>> 1, so the release manager (who happened to be me) knew
> that
> > the
> > >> > KIP
> > >> > > >>>>>> would
> > >> > > >>>>>>> not make it in this release and would need to be postponed
> > to a
> > >> > > future
> > >> > > >>>>>>> release. After that, development on this feature continued
> > and
> > >> it
> > >> > > was
> > >> > > >>>>>>> declared to enter level 2 right in time for being in Kafka
> > 3.9.
> > >> > > >>>>>>>
> > >> > > >>>>>>> Let me know what you think.
> > >> > > >>>>>>>
> > >> > > >>>>>>> Best,
> > >> > > >>>>>>>
> > >> > > >>>>>>> On Mon, Aug 19, 2024 at 8:51 AM TengYao Chi <
> > >> kiting...@gmail.com
> > >> > >
> > >> > > >>>>>> wrote:
> > >> > > >>>>>>>
> > >> > > >>>>>>>> Hello Josep,
> > >> > > >>>>>>>> I think this KIP is a great addition to the community
> that
> > we
> > >> > now
> > >> > > >>>>>> have a
> > >> > > >>>>>>>> crystal-clear definition for the state of a feature.
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> In the current proposal, I think Level 1 is defined as
> the
> > >> stage
> > >> > > >>>>>> where a
> > >> > > >>>>>>>> feature is "incomplete and unusable", while Level 2
> > >> represents a
> > >> > > >>>>>> feature
> > >> > > >>>>>>>> that is "usable but potentially incomplete".
> > >> > > >>>>>>>> The distinction between these two levels mig
> > <
> https://www.google.com/maps/search/on+between+these+two+levels+mig?entry=gmail&source=g
> >ht
> > not always be
> > >> > > clear,
> > >> > > >>>>>>>> especially during the transition of a feature from
> > "unusable"
> > >> to
> > >> > > >>>>>> "usable
> > >> > > >>>>>>>> but incomplete".
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> IMHO, to simplify the process and reduce confusion for
> both
> > >> > > >>> developers
> > >> > > >>>>>>> and
> > >> > > >>>>>>>> users, I would suggest merging Level 1 and Level 2 into a
> > >> single
> > >> > > >>>>>> unified
> > >> > > >>>>>>>> level.
> > >> > > >>>>>>>> This merged level could cover the entire phase from when
> a
> > >> > > feature is
> > >> > > >>>>>>>> unstable to when it becomes usable but incomplete.
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> WYDT?
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> Best regards,
> > >> > > >>>>>>>> TengYao
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> Josep Prat <josep.p...@aiven.io.invalid> 於 2024年8月19日 週一
> > >> > > 上午2:58寫道：
> > >> > > >>>>>>>>
> > >> > > >>>>>>>>> Hi Chia-Ping,
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>> As far as I can tell, Tiered Storage is still at level
> 3.
> > I
> > >> > think
> > >> > > >>>>>> the
> > >> > > >>>>>>>>> intention would be to declare it level 4 in 4.0.0.
> > >> > > >>>>>>>>> KIP-848 was in level 2 in Kafka 3.7. and it went level 3
> > in
> > >> > Kafka
> > >> > > >>>>>> 3.8.
> > >> > > >>>>>>>>> Level 4 features would be for example MirrorMaker2 for
> > >> example.
> > >> > > As
> > >> > > >>>>>> far
> > >> > > >>>>>>>> as I
> > >> > > >>>>>>>>> understand the Docker image is level 4.
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>> Does that make sense? If so I can update the KIP with
> > those
> > >> > > >>>>>> examples.
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>> Best,
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>> ------------------
> > >> > > >>>>>>>>> Josep Prat
> > >> > > >>>>>>>>> Open Source Engineering Director, Aiven
> > >> > > >>>>>>>>> josep.p...@aiven.io | +491715557497 | aiven.io
> > >> > > >>>>>>>>> Aiven Deutschland GmbH
> > >> > > >>>>>>>>> Alexanderufer 3-7, 10117 Berlin
> > >> > > >>>>>>>>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
> > >> > > >>>>>>>>> Anna Richardson, Kenneth Chen
> > >> > > >>>>>>>>> Amtsgericht Charlottenburg, HRB 209739 B
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>> On Sun, Aug 18, 2024, 21:46 Chia-Ping Tsai <
> > >> chia7...@gmail.com
> > >> > >
> > >> > > >>>>>> wrote:
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>>> hi Josep
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> Although I didn't join the discussion before, the KIP
> is
> > >> > > >>>>>> interesting
> > >> > > >>>>>>>> and
> > >> > > >>>>>>>>>> great to me.
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> one small comment:
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> Could you please add existent features as an example to
> > each
> > >> > > level
> > >> > > >>>>>>> for
> > >> > > >>>>>>>>> the
> > >> > > >>>>>>>>>> readers who have poor reading (like me) ? For
> instance, I
> > >> > guess
> > >> > > >>>>>> the
> > >> > > >>>>>>> new
> > >> > > >>>>>>>>>> coordinator is level 3? tiered storage is level 4?
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> Best,
> > >> > > >>>>>>>>>> Chia-Ping
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> Josep Prat <josep.p...@aiven.io.invalid> 於 2024年8月19日
> 週一
> > >> > > >>>>>> 上午2:13寫道：
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>>> Hi all,
> > >> > > >>>>>>>>>>> I want to start a discussion for KIP-1081: Graduation
> > Steps
> > >> > for
> > >> > > >>>>>>>>> Features.
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>
> > >> > > >>>
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1081%3A+Graduation+Steps+for+Features
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> We already had a bit of a discussion here
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>
> > >> https://lists.apache.org/thread/5z6rxvs9m0bro5ssmtg8qcgdk40882bz
> > >> > > >>>>>>> and
> > >> > > >>>>>>>>>> that
> > >> > > >>>>>>>>>>> materialized into this KIP.
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> I deliberately defined the graduation steps without
> > giving
> > >> > them
> > >> > > >>>>>> a
> > >> > > >>>>>>>> name,
> > >> > > >>>>>>>>>> so
> > >> > > >>>>>>>>>>> we don't go bike-shedding there. There is a separate
> > >> section
> > >> > > for
> > >> > > >>>>>>> the
> > >> > > >>>>>>>>>> names
> > >> > > >>>>>>>>>>> of each step. Also an alternative set of names. I'd
> > like to
> > >> > get
> > >> > > >>>>>>> some
> > >> > > >>>>>>>>>>> feedback on the steps, and also on the names for the
> > steps.
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> Looking forward to your opinions, and hopefully only a
> > tiny
> > >> > bit
> > >> > > >>>>>> of
> > >> > > >>>>>>>>>>> bike-shedding :)
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> Best,
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> --
> > >> > > >>>>>>>>>>> [image: Aiven] <https://www.aiven.io/>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> *Josep Prat*
> > >> > > >>>>>>>>>>> Open Source Engineering Director, *Aiven*
> > >> > > >>>>>>>>>>> josep.p...@aiven.io | +491715557497
> > >> > > >>>>>>>>>>> aiven.io <https://www.aiven.io/> | <
> > >> > > >>>>>>>>>> https://www.facebook.com/aivencloud
> > >> > > >>>>>>>>>>>>
> > >> > > >>>>>>>>>>> <https://www.linkedin.com/company/aiven/> <
> > >> > > >>>>>>>>>>> https://twitter.com/aiven_io>
> > >> > > >>>>>>>>>>> *Aiven Deutschland GmbH*
> > >> > > >>>>>>>>>>> Alexanderufer 3-7, 10117 Berlin
> > >> > > >>>>>>>>>>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
> > >> > > >>>>>>>>>>> Anna Richardson, Kenneth Chen
> > >> > > >>>>>>>>>>> Amtsgericht Charlottenburg, HRB 209739 B
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>>
> > >> > > >>>>>>> --
> > >> > > >>>>>>> [image: Aiven] <https://www.aiven.io/>
> > >> > > >>>>>>>
> > >> > > >>>>>>> *Josep Prat*
> > >> > > >>>>>>> Open Source Engineering Director, *Aiven*
> > >> > > >>>>>>> josep.p...@aiven.io | +491715557497
> > >> > > >>>>>>> aiven.io <https://www.aiven.io/> | <
> > >> > > >>>>>> https://www.facebook.com/aivencloud
> > >> > > >>>>>>>>
> > >> > > >>>>>>> <https://www.linkedin.com/company/aiven/> <
> > >> > > >>>>>>> https://twitter.com/aiven_io>
> > >> > > >>>>>>> *Aiven Deutschland GmbH*
> > >> > > >>>>>>> Alexanderufer 3-7, 10117 Berlin
> > >> > > >>>>>>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
> > >> > > >>>>>>> Anna Richardson, Kenneth Chen
> > >> > > >>>>>>> Amtsgericht Charlottenburg, HRB 209739 B
> > >> > > >>>>>>>
> > >> > > >>>>>>
> > >> > > >>>>>
> > >> > > >>>>>
> > >> > > >>>>> --
> > >> > > >>>>> [image: Aiven] <https://www.aiven.io/>
> > >> > > >>>>>
> > >> > > >>>>> *Josep Prat*
> > >> > > >>>>> Open Source Engineering Director, *Aiven*
> > >> > > >>>>> josep.p...@aiven.io | +491715557497
> > >> > > >>>>> aiven.io <https://www.aiven.io/> |
> > >> > > >>>>> <https://www.facebook.com/aivencloud>
> > >> > > >>>>> <https://www.linkedin.com/company/aiven/> <
> > >> > > >>> https://twitter.com/aiven_io>
> > >> > > >>>>> *Aiven Deutschland GmbH*
> > >> > > >>>>> Alexanderufer 3-7, 10117 Berlin
> > >> > > >>>>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
> > >> > > >>>>> Anna Richardson, Kenneth Chen
> > >> > > >>>>> Amtsgericht Charlottenburg, HRB 209739 B
> > >> > > >>>>>
> > >> > > >>>>
> > >> > > >>>>
> > >> > > >>>
> > >> > > >>
> > >> > > >>
> > >> > > >> --
> > >> > > >> [image: Aiven] <https://www.aiven.io/>
> > >> > > >>
> > >> > > >> *Josep Prat*
> > >> > > >> Open Source Engineering Director, *Aiven*
> > >> > > >> josep.p...@aiven.io | +491715557497
> > >> > > >> aiven.io <https://www.aiven.io/> | <
> > >> > > https://www.facebook.com/aivencloud>
> > >> > > >> <https://www.linkedin.com/company/aiven/> <
> > >> > > https://twitter.com/aiven_io>
> > >> > > >> *Aiven Deutschland GmbH*
> > >> > > >> Alexanderufer 3-7, 10117 Berlin
> > >> > > >> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
> > >> > > >> Anna Richardson, Kenneth Chen
> > >> > > >> Amtsgericht Charlottenburg, HRB 209739 B
> > >> > >
> > >> >
> > >> >
> > >> > --
> > >> > David Arthur
> > >> >
> > >>
> > >>
> > >> --
> > >> [image: Aiven] <https://www.aiven.io>
> > >>
> > >> *Josep Prat*
> > >> Open Source Engineering Director, *Aiven*
> > >> josep.p...@aiven.io | +491715557497
> > >> aiven.io <https://www.aiven.io> | <
> > https://www.facebook.com/aivencloud
> > >> >
> > >> <https://www.linkedin.com/company/aiven/> <
> > >> https://twitter.com/aiven_io>
> > >> *Aiven Deutschland GmbH*
> > >> Alexanderufer 3-7, 10117 Berlin
> > >> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen,
> > >> Anna Richardson, Kenneth Chen
> > >> Amtsgericht Charlottenburg, HRB 209739 B
> > >>
> >
>
