Hi Kamal, It is fine to relax the constraint in StructSpec for your internal tags. For instance, you could allow your internal tags to start from id 10000. It is very unlikely that we will ever have tags as part of the official protocol reaching that.
Best, David On Mon, Feb 3, 2025 at 6:36 PM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Bumping up on this thread. > > In a forked Kafka repo, if we want to introduce a new tagged field. Which > approach should be taken? > Is it fine to relax the constraint in StructSpec > < > https://github.com/apache/kafka/blob/trunk/generator/src/main/java/org/apache/kafka/message/StructSpec.java#L76 > > > to > start with a very large tag-id like 10000 for custom > tagged fields to not clash with future tagged fields being introduced > upstream? TIA. > > Thanks, > Kamal > > On Thu, Jan 23, 2025 at 11:47 PM Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: > > > Hi all, > > > > In KIP-482 > > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-482%3A+The+Kafka+Protocol+should+Support+Optional+Tagged+Fields#KIP482:TheKafkaProtocolshouldSupportOptionalTaggedFields-FlexibleVersions > >, > > we have introduced support for optional tagged fields when the > > API-message supports "flexibleVersions" feature. One of the motivations > of > > the KIP is to support attaching extra information to a message without > > requiring a version bump for everyone who has to read that message. > > > > In StructSpec > > < > https://github.com/apache/kafka/blob/trunk/generator/src/main/java/org/apache/kafka/message/StructSpec.java#L76 > >, > > there is a constraint to start the tagId of the tagged field from 0 and > > should be contiguous. Once the user introduces a custom taggedField that > > starts from 0, there is a risk of protocol divergence between the server > > and client when the upstream/Apache introduces a new taggedField in > future > > that starts from 0. > > > > Once a tag-id gets introduced, it cannot be used for anything else. We > > want to propagate a custom trace-id in the API-message between > > server/client and want to use a large tagId like 10001 (2 bytes varint) > to > > avoid conflict with the future API message specification. > > > > We would like to understand the reason behind the tagId constraint to > > start from 0 and contiguous. One use-case of having this constraint > ensures > > that the tag-id starts from 0 and won't be random. If it is random, then > > the user won't be sure in defining the unique tagged-ids for their > > user-defined fields. > > > > I’d appreciate it if you could take a look and share your thoughts or > > feedback. > > > > Thanks, > > Kamal > > > > > > > > >
