Hey folks, Wanted to provide one more update for KIP-1022. There was some discussion about feature versions set by default vs features that were considered production ready. In the current implementation, those are both controlled by the metadata version mapping for each feature. After some discussion with folks working on group version, we thought it would make sense to have two separate mechanisms. One to mark production readiness/stability and one to set the default.
We intend to keep the default setting as is (marking the corresponding MV as production ready), but for feature level production readiness, we will introduce a field to each feature to indicate the latest production version. This allows setting the feature via --feature without unstable.feature.versions.enable but without making the feature default yet. There is an open PR to show how this would work. https://github.com/apache/kafka/pull/17886 Let me know if folks have any questions or comments. Thanks, Justine On Tue, Oct 1, 2024 at 10:57 AM Justine Olshan <jols...@confluent.io> wrote: > Thanks Jiunn-Yang for the update. It makes sense to me. > > I also wanted to provide my own update. While working on KAFKA-16308, we > ran into a few issues with feature validation. Thus we made the following > changes to the UpdateFeaturesResponse > > > 1. Any update request will fail fast if a single feature fails to > update. This error will become the top level error in the response, and no > feature level responses will be returned. The top level error message > contains the feature that failed. If there is any error, none of the > updates will persist. > 2. For V2 requests and above, successful updates (NONE error code) > will also not contain the results field. > > > I have added this information to the KIP. Let me know if there are any > questions or concerns. > > Justine > > On Tue, Sep 24, 2024 at 7:07 AM 黃竣陽 <s7133...@gmail.com> wrote: > >> Hello Justine, >> Thank you for your feedback. >> >> The strikethrough sections in the KIP reflect an approach we initially >> considered but later decided against. To make it clearer for future >> readers, I will revise this part to explain the change in direction using >> words. >> >> Thanks for your advise, and feel free to share any further suggestions. >> >> Best regards, >> Jiunn-Yang >> >> >> > Justine Olshan <jols...@confluent.io.INVALID> 於 2024年9月16日 晚上10:39 寫道： >> > >> > Hey there Jiunn-Yang, >> > >> > Thanks for taking the time to update the KIP. I noticed the section now >> has >> > some crossed out components. I am a little confused since I think this >> > means we originally took a certain approach, but decided against it >> later? >> > Is it possible to explain we took one approach but decided against it >> later >> > in words and not crossing out text? I think it will help anyone who >> reads >> > this KIP in the future understand. >> > >> > Thanks again, >> > Justine >> > >> > On Sat, Sep 14, 2024 at 6:21 PM 黃竣陽 <s7133...@gmail.com> wrote: >> > >> >> Hi everyone, >> >> >> >> I make some update to KIP-1022 < >> >> >> >> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1022%3A+Formatting+and+Updating+Features >> >>> >> >> As part of KIP-584, brokers expose supported version ranges for >> features, >> >> which may include 0. Version 0 can indicate either an incomplete >> feature or >> >> a disabled one. However, older software versions have an issue in >> >> SupportedVersionRange that prevents handling ranges that include 0, >> leading >> >> to deserialization problems in ApiVersionsResponse. >> >> One approach is to modify the minimum version from 0 to 1, which >> passes old >> >> client validation but removes the ability to disable the feature. >> Another >> >> approach is to omit features with min version 0 in requests. >> >> KAFKA-17011 implements the solution of modifying the minimum version >> from 0 >> >> to 1 for RPC versions v0 to v3. This passes old client validation but >> >> removes the ability to disable the feature. >> >> A related issue occurs when a new broker sends a >> BrokerRegistrationRequest >> >> to an old controller, as the old controller does not support version >> 1. The >> >> solution is to omit features with version 0 in requests for versions >> v0 to >> >> v3. Therefore, we need to adjust the approach to resolve this dilemma. >> >> In KAFKA-17492, we omit the minimum version of 0 for RPC versions v0 >> to v3, >> >> allowing a 3.9 broker to join a 3.8 broker controller. >> >> >> >> Please let me know if you have any questions or concerns regarding this >> >> approach. >> >> Thanks you, >> >> Best regard, >> >> Jiunn-Yang >> >> >> >> On 2024/04/02 18:49:50 Justine Olshan wrote: >> >>> Hi José >> >>> >> >>> I originally had each on a new line and then switched to a single line >> >>> since it looked like a lot of space. I can switch it back. >> >>> >> >>> I don't think it makes a big difference if we implement it for both >> >> tools. >> >>> Both tools will have use for it. >> >>> >> >>> I can change the name to feature-dependencies if that makes it >> clearer. >> >>> >> >>> I can say that we won't allow cycles, but I'm not sure the best way to >> >>> enforce this. >> >>> >> >>> I just put ">" to show output. I can change the color if that makes it >> >>> clearer. >> >>> >> >>> Justine >> >>> >> >>> On Tue, Apr 2, 2024 at 11:41 AM José Armando García Sancio >> >>> <js...@confluent.io.invalid> wrote: >> >>> >> >>>> Hi Justine, >> >>>> >> >>>> See my comments below. >> >>>> >> >>>> On Mon, Apr 1, 2024 at 4:43 PM Justine Olshan >> >>>> <jo...@confluent.io.invalid> wrote: >> >>>>> 20. I can update the KIP. >> >>>> >> >>>> I took a look at the latest KIP. >> >>>> >> >>>> * Should the output of this command "bin/kafka-features.sh >> >>>> version-mapping --release-version 3.6-IV1" be something like this: >> >>>> metadata.version=13 >> >>>> transaction.protocol.version=0 >> >>>> group.coordinator.version=0 >> >>>> kraft.version=0 >> >>>> >> >>>> I added the kraft.version to show the changes that are coming from >> >>>> KIP-853. I find this formatting much easier to parse by >> scripts/tools. >> >>>> They can even use Java's Properties parser if they want. >> >>>> >> >>>> * Maybe I missed the discussion but can you talk about why both >> >>>> "kafka-storage" and "kafka-features" are going to implement the >> >>>> "version-mapping" and dependencies"? I assume that it is sufficient >> >>>> for only one (kafka-features) to implement these new commands. >> >>>> >> >>>> * For the command "dependencies" in the "kafka-features" command, it >> >>>> is probably obvious that the dependencies listed are feature version >> >>>> dependencies. This is not obvious when the user uses "kafka-storage >> >>>> dependencies". This reads as the dependencies of kafka-storage. >> >>>> >> >>>> * Should we state that Kafka will not allow cycles in the feature >> >>>> version dependency definition? Meaning the user is not allowed to >> >>>> define a feature version X which depends on Y which depends on Z >> which >> >>>> depends on X. >> >>>> >> >>>> * Some of the example output start with the characters "> ". Will the >> >>>> CLI print those characters or is that just part of the KIP's >> >>>> formating? >> >>>> >> >>>> Thanks, >> >>>> -- >> >>>> -José >> >>>> >> >>> >> >> >> >>
