Re: [VOTE] KIP-1022 Formatting and Updating Features

Hmm, I thought I had added that already. I guess I missed it. Sorry for the confusion, and thanks for the update. best, Colin On Tue, Jul 30, 2024, at 15:06, Jun Rao wrote: > Thanks for updating the KIP, Justine. > > Jun > > On Tue, Jul 30, 2024 at 1:37 PM Justine Olshan > wrote: > >> I added

Re: [VOTE] KIP-1022 Formatting and Updating Features

Thanks for updating the KIP, Justine. Jun On Tue, Jul 30, 2024 at 1:37 PM Justine Olshan wrote: > I added this update to the end of the section Colin added. > > Justine > > On Tue, Jul 30, 2024 at 11:01 AM Jun Rao wrote: > > > Hi, Colin, > > > > Thanks for the update. We also excluded supporte

Re: [VOTE] KIP-1022 Formatting and Updating Features

I added this update to the end of the section Colin added. Justine On Tue, Jul 30, 2024 at 11:01 AM Jun Rao wrote: > Hi, Colin, > > Thanks for the update. We also excluded supported features with maxVersion > of 0 from both ApiVersionResponse and BrokerRegistrationRequest, and > excluded finali

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi, Colin, Thanks for the update. We also excluded supported features with maxVersion of 0 from both ApiVersionResponse and BrokerRegistrationRequest, and excluded finalized features with version of 0 from ApiVersionResponse. It would be useful to document those too. Jun On Mon, Jul 29, 2024 at

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi Jun, Just to close the loop on this... the KIP now mentions both ApiVersionResponse and BrokerRegistrationRequest. best, Colin On Mon, Jul 8, 2024, at 14:57, Jun Rao wrote: > Hi, Colin, > > Thanks for the update. Since the PR also introduces a new version of > BrokerRegistrationRequest, coul

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi, Colin, Thanks for the update. Since the PR also introduces a new version of BrokerRegistrationRequest, could we include that change in the KIP update too? Jun On Mon, Jul 8, 2024 at 11:08 AM Colin McCabe wrote: > Hi all, > > I've updated the approach in https://github.com/apache/kafka/pull

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi all, I've updated the approach in https://github.com/apache/kafka/pull/16421 so that we change the minVersion=0 to minVersion=1 in older ApiVersionsResponses. I hope we can get this in soon and unblock the features that are waiting for it! best, Colin On Wed, Jul 3, 2024, at 10:55, Jun Rao

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi, David, Thanks for the reply. In the common case, there is no difference between omitting just v0 of the feature or omitting the feature completely. It's just when an old client is used, there is some difference. To me, omitting just v0 of the feature seems slightly better for the old client.

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi Jun, Colin, Thanks for your replies. If the FeatureCommand relies on version 0 too, my suggestion does not work. Omitting the features for old clients as suggested by Colin seems fine for me. In practice, administrators will usually use a version of FeatureCommand matching the cluster version

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi David, In the ApiVersionsResponse, we really don't have an easy way of mapping finalizedVersion = 1 to "off" in older releases such as 3.7.0. For example, if a 3.9.0 broker advertises that it has finalized group.version = 1, that will be treated by 3.7.0 as a brand new feature, not as "KIP-8

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi, David, Yes, that's another option. It probably has its own challenges. For example, the FeatureCommand tool currently treats disabling a feature as setting the version to 0. It would be useful to get Jose's opinion on this since he introduced version 0 in the kraft.version feature. Thanks, J

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi Jun, Colin, Have we considered sticking with the range going from version 1 to N where version 1 would be the equivalent of "disabled"? In the group.version case, we could introduce group.version=1 that does basically nothing and group.version=2 that enables the new protocol. I suppose that we

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi, Colin, Yes, #3 is the scenario that I was thinking about. In either approach, there will be some information missing in the old client. It seems that we should just pick the one that's less wrong. In the more common case when a feature is finalized on the server, presenting a supported featur

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi Jun, This is a fair question. I think there's a few different scenarios to consider: 1. mixed server software versions in a single cluster 2. new client software + old server software 3. old client software + new server software In scenario #1 and #2, we have old (pre-3.9) server software i

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi, Colin, ApiVersionResponse includes both supported and finalized features. If we only suppress features in the supported field, but not in the finalized field, it can potentially lead to inconsistency in the older client. For example, if a future feature supporting V0 is finalized in the broker

Re: [VOTE] KIP-1022 Formatting and Updating Features

Thanks for the reply, Justine and Colin. Sounds good to me. Jun On Wed, Jun 26, 2024 at 12:54 PM Colin McCabe wrote: > Hi Justine, > > Yes, that was what I was thinking. > > best, > Colin > > On Mon, Jun 24, 2024, at 11:11, Justine Olshan wrote: > > My understanding is that the tools that don't

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi Justine, Yes, that was what I was thinking. best, Colin On Mon, Jun 24, 2024, at 11:11, Justine Olshan wrote: > My understanding is that the tools that don't rely on ApiVersions should > still return 0s when it is the correct value. I believe these commands do > not require this API and thus

Re: [VOTE] KIP-1022 Formatting and Updating Features

My understanding is that the tools that don't rely on ApiVersions should still return 0s when it is the correct value. I believe these commands do not require this API and thus can show 0 as versions. Likewise, when the old ApiVersionsRequest is used to describe features, we can't return 0 version

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi, Colin, Thanks for the update. The proposed change seems reasonable to me. Just one clarification. The KIP can show version 0 of certain features with version-mapping and feature-dependencies. Will that part change? For example, will the tool show version 0 features with --release-version 3.8

Re: [VOTE] KIP-1022 Formatting and Updating Features

Thanks for the update Colin. The changes make sense to me. Are you planning to update the KIP to reflect this new RPC version? It would be good to document the semantics explained above in the KIP. Thanks! On Fri, Jun 21, 2024 at 8:22 PM Justine Olshan wrote: > > Ok makes sense. I will update m

Re: [VOTE] KIP-1022 Formatting and Updating Features

Ok makes sense. I will update my PR. On Fri, Jun 21, 2024 at 5:09 PM Colin McCabe wrote: > I think it's better to suppress the response in v3. The issue with > modifying it is that there may be scenarios where [1, 1] is the actual > supported range, and we'd want to know that. But leaving out th

Re: [VOTE] KIP-1022 Formatting and Updating Features

I think it's better to suppress the response in v3. The issue with modifying it is that there may be scenarios where [1, 1] is the actual supported range, and we'd want to know that. But leaving out the feature should be OK for older clients (it will be the case with clients old enough to send a

Re: [VOTE] KIP-1022 Formatting and Updating Features

Thanks Colin, This makes sense to me. Namely in the case where we perhaps don't want to support version 0 anymore, we need the range to be able to not include 0. (In other words, we can't assume 0 is supported) It is unfortunate that this change is a bit tricky, but I think it's the best option.

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi all, It seems that there was a bug in older versions of Kafka which caused deserialization problems when a supported feature range included 0. For example, the range for group.version of [0, 1] would be a problem in this situation. This obviously makes supportedVersions kind of useless. Any

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hey folks, Thanks everyone! I will go ahead and call it. The KIP passes with the following +1 votes: - Andrew Schofield (non-binding) - David Jacot (binding) - José Armando García Sancio (binding) - Jun Rao (binding) Thanks again, Justine On Fri, Apr 12, 2024 at 11:16 AM Jun Rao wrote: > Hi,

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi, Justine, Thanks for the KIP. +1 Jun On Wed, Apr 10, 2024 at 9:13 AM José Armando García Sancio wrote: > Hi Justine, > > +1 (binding) > > Thanks for the improvement. > -- > -José >

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi Justine, +1 (binding) Thanks for the improvement. -- -José

Re: [VOTE] KIP-1022 Formatting and Updating Features

+1 (binding). Thanks for the KIP! On Mon, Apr 8, 2024 at 7:23 PM Andrew Schofield < andrew_schofield_j...@outlook.com> wrote: > Hi Justine, > Thanks for the KIP. > > +1 (non-binding) > > Thanks, > Andrew > > > On 8 Apr 2024, at 18:07, Justine Olshan > wrote: > > > > Hello all, > > I would like t

Re: [VOTE] KIP-1022 Formatting and Updating Features

Hi Justine, Thanks for the KIP. +1 (non-binding) Thanks, Andrew > On 8 Apr 2024, at 18:07, Justine Olshan wrote: > > Hello all, > I would like to start a vote for KIP-1022 Formatting and Updating Features >

[VOTE] KIP-1022 Formatting and Updating Features

Hello all, I would like to start a vote for KIP-1022 Formatting and Updating Features Please take a look and cast your vote. Thanks, Justine