Got it. Thanks for clearing that up! -- Kamal
On Tue, Feb 4, 2025 at 4:40 PM David Jacot <dja...@confluent.io.invalid> wrote: > Hi Kamal, > > It is fine to relax the constraint in StructSpec for your internal tags. > For instance, you could allow your internal tags to start from id 10000. It > is very unlikely that we will ever have tags as part of the official > protocol reaching that. > > Best, > David > > On Mon, Feb 3, 2025 at 6:36 PM Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: > > > Bumping up on this thread. > > > > In a forked Kafka repo, if we want to introduce a new tagged field. Which > > approach should be taken? > > Is it fine to relax the constraint in StructSpec > > < > > > https://github.com/apache/kafka/blob/trunk/generator/src/main/java/org/apache/kafka/message/StructSpec.java#L76 > > > > > to > > start with a very large tag-id like 10000 for custom > > tagged fields to not clash with future tagged fields being introduced > > upstream? TIA. > > > > Thanks, > > Kamal > > > > On Thu, Jan 23, 2025 at 11:47 PM Kamal Chandraprakash < > > kamal.chandraprak...@gmail.com> wrote: > > > > > Hi all, > > > > > > In KIP-482 > > > < > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-482%3A+The+Kafka+Protocol+should+Support+Optional+Tagged+Fields#KIP482:TheKafkaProtocolshouldSupportOptionalTaggedFields-FlexibleVersions > > >, > > > we have introduced support for optional tagged fields when the > > > API-message supports "flexibleVersions" feature. One of the motivations > > of > > > the KIP is to support attaching extra information to a message without > > > requiring a version bump for everyone who has to read that message. > > > > > > In StructSpec > > > < > > > https://github.com/apache/kafka/blob/trunk/generator/src/main/java/org/apache/kafka/message/StructSpec.java#L76 > > >, > > > there is a constraint to start the tagId of the tagged field from 0 and > > > should be contiguous. Once the user introduces a custom taggedField > that > > > starts from 0, there is a risk of protocol divergence between the > server > > > and client when the upstream/Apache introduces a new taggedField in > > future > > > that starts from 0. > > > > > > Once a tag-id gets introduced, it cannot be used for anything else. We > > > want to propagate a custom trace-id in the API-message between > > > server/client and want to use a large tagId like 10001 (2 bytes varint) > > to > > > avoid conflict with the future API message specification. > > > > > > We would like to understand the reason behind the tagId constraint to > > > start from 0 and contiguous. One use-case of having this constraint > > ensures > > > that the tag-id starts from 0 and won't be random. If it is random, > then > > > the user won't be sure in defining the unique tagged-ids for their > > > user-defined fields. > > > > > > I’d appreciate it if you could take a look and share your thoughts or > > > feedback. > > > > > > Thanks, > > > Kamal > > > > > > > > > > > > > > >
