Hi, Andrew, Thanks for the reply.
JR6. It seems weird that the implementation of a KIP-714 plugin requires casting an authorizableRequestContext to an internal class RequestContext. Would it be better to expose all client related resource labels through a public API in ClientTelemetryContext? Jun On Mon, Sep 21, 2026 at 6:43 AM Andrew Schofield <[email protected]> wrote: > Hi Jun, > Thanks for the reply. > > JR4: Done. Producer, consumer and admin. > > JR5: Yes. I've reworded slightly. > > JR6: I think the plugin implementation already needs to do something > similar to get to software name/version. > ClientTelemetryContext.authorizableRequestContext() returns an instance of > AuthorizableRequestContext. The o.a.k.common.requests.RequestContext class > implements that interface, and it also lets you get to ClientInformation > and that's where you can find the software name/version and framework > name/version. You can see this in > o.a.k.server.metrics.ClientMetricsInstanceMetadata. I would describe this > as grubby :) > > One way forward here would be to remove the broker-added resource labels > from KIP-1368, but leave the additions to the match criteria. Then it would > still be possible to specify, for example, that particular metrics should > be captured for Kafka Streams clients only, without adding to the > grubbiness. > > Thanks, > Andrew > > On 2026/09/15 20:44:06 Jun Rao via dev wrote: > > Hi, Andrew, > > > > Thanks for the reply. A few more comments. > > > > JR4. "The configuration keys are defined in > > org.apache.kafka.clients.CommonClientConfigs" > > Could you explicitly list the clients (producer, consumer, AdminClient, > > etc.) that define the two new configs? > > > > JR5. Should AK frameworks such as kstream and connect set > > client.framework.name and client.framework.version in their clients > > automatically? > > > > JR6. "Broker-added resources labels for client metrics" > > Are the new labels added in DefaultClientTelemetryContext.RequestContext? > > Since ClientTelemetryExporter.exportMetrics() only takes > > ClientTelemetryContext, does the implementor need to cast it > > to DefaultClientTelemetryContext to retrieve the new labels? > > > > Jun > > > > On Fri, Sep 11, 2026 at 12:48 PM Andrew Schofield <[email protected] > > > > wrote: > > > > > Hi Jun, > > > JR1: I dug into the KIP-714 client metrics stuff a bit more and I have > > > revised the KIP. I have added client-framework-name and > > > client-framework-version to the set of keys in the matching for client > > > metrics, and also added these broker-added metrics tags. This is how > these > > > pieces of information would have been incorporated into KIP_714 had > they > > > been part of Kafka at that point. > > > > > > Thanks, > > > Andrew > > > > > > On 2026/08/04 20:29:19 Andrew Schofield wrote: > > > > Hi Jun, > > > > Thanks for the response. > > > > > > > > JR1: My intent here is only request logging. It would be possible to > add > > > client-framework-name and client-framework-version to the match > criteria > > > for client metrics, which could conceivably permit scenarios such as > > > capturing metrics from specific versions of the Spring framework. This > is > > > beyond the scope I have in mind. > > > > > > > > JR3: Thanks for clarifying the tagged field conventions. Version bump > > > reverted. > > > > > > > > Thanks, > > > > Andrew > > > > > > > > On 2026/08/04 18:38:40 Jun Rao via dev wrote: > > > > > Hi, Andrew, > > > > > > > > > > Thanks for the reply. > > > > > > > > > > JR1. ClientSoftwareName and ClientSoftwareVersion are used in > metric > > > names > > > > > and match predicates for configuring client metrics. Are > > > ClientFrameworkName > > > > > and ClientFrameworkVersion used in those places too or are they > only > > > used > > > > > in request logging? > > > > > > > > > > JR3. The general rule for adding a new field in RPC is that (1) if > it's > > > > > truly optional, we add it as a tagged field with no version bump; > (2) > > > > > otherwise, we add it as a non-tagged field with a version bump. One > > > > > exception is when adding a non-optional field in the request > header. > > > > > Because of the limitation in existing implementation, we need to > add > > > it as > > > > > a tagged field with a version bump. > > > > > > > > > > Jun > > > > > > > > > > On Tue, Aug 4, 2026 at 6:04 AM Andrew Schofield < > [email protected] > > > > > > > > > wrote: > > > > > > > > > > > Hi Jun, > > > > > > Thanks for your response. > > > > > > > > > > > > JR1: I've expanded the motivation section in the KIP. They are > truly > > > > > > optional, I feel. When diagnosing an issue using the client logs, > > > having > > > > > > this information can illuminate why the application is behaving > in a > > > > > > particular way. If the application is using a framework, > behaviours > > > such as > > > > > > retries will typically not be in the application code itself > because > > > > > > they're implemented in the framework. There's a difference > between > > > what the > > > > > > application developer thinks their code does and what we see in > the > > > logs. > > > > > > That's the point. > > > > > > > > > > > > JR2: Done. They're Type.STRING, default null. I think they > should be > > > > > > importance LOW because that affects the prominence of the > configs in > > > the > > > > > > documentation, but I wonder whether you agree. > > > > > > > > > > > > Thanks, > > > > > > Andrew > > > > > > > > > > > > On 2026/08/03 21:39:06 Jun Rao via dev wrote: > > > > > > > Hi, Andrew, > > > > > > > > > > > > > > Thanks for the KIP. > > > > > > > > > > > > > > JR1. Could you describe the use cases of the two new configs > and > > > are they > > > > > > > truly optional? > > > > > > > > > > > > > > JR2. Could you add the type and the default value for the new > > > configs? > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > On Fri, Jul 31, 2026 at 10:23 AM Matthias J. Sax < > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Thanks for the KIP Andrew. I think it will be very useful if > we > > > can > > > > > > > > identify frameworks, especially our own ones (Connect and > > > Streams). > > > > > > > > > > > > > > > > About Aditya's point: I am wondering where to draw the line. > Many > > > > > > > > examples seems to be metadata that belongs into he Kafka > record > > > > > > > > `Headers` at the application level, rather than the lower > level > > > request > > > > > > > > headers? > > > > > > > > > > > > > > > > There if of course no strict logical dividing line between > both. > > > And > > > > > > > > yes, the broker does not access application level Kafka > record > > > > > > > > `Headers`. But if it would be useful to let the broker tap > into > > > > > > > > application level record `Headers` we should tackle it > > > independently? > > > > > > > > > > > > > > > > Personally, I don't think it would be the right design to > push > > > too many > > > > > > > > thing into the lower level request headers. So I am in favor > of > > > only > > > > > > > > added the two new propose `ClientFrameworkName` and > > > > > > > > `ClientFrameworkVersion` fields. > > > > > > > > > > > > > > > > > > > > > > > > -Matthias > > > > > > > > > > > > > > > > On 7/31/26 7:56 AM, Federico Valeri wrote: > > > > > > > > > Changes look good. Thanks. > > > > > > > > > > > > > > > > > > On Fri, Jul 31, 2026 at 11:17 AM Andrew Schofield < > > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > >> > > > > > > > > >> Hi Fede, > > > > > > > > >> Thanks for your response. > > > > > > > > >> > > > > > > > > >> FV1: They are set using the regular config properties. > Yes, > > > it is > > > > > > > > possible for an end user to set arbitrary values, but then > that > > > would > > > > > > also > > > > > > > > be true of a builder or internal constructor. I've added a > bit > > > more > > > > > > > > information in the KIP and beefed up the config descriptions > to > > > > > > discourage > > > > > > > > application use. > > > > > > > > >> > > > > > > > > >> FV2: I've never encountered such nightmares myself. The > > > framework > > > > > > which > > > > > > > > sets the config last would win. Maybe this is a motivation > for > > > having a > > > > > > > > programmatic way of setting the information. We could support > > > > > > concatenation > > > > > > > > of framework information, but that's just pandering to these > > > people. > > > > > > Let me > > > > > > > > know what you think. > > > > > > > > >> > > > > > > > > >> I've also updated the KIP with a maximum length for these > > > pieces of > > > > > > > > information since all identifiers should have defined bounds. > > > > > > > > >> > > > > > > > > >> Thanks, > > > > > > > > >> Andrew > > > > > > > > >> > > > > > > > > >> On 2026/07/31 08:50:11 Federico Valeri wrote: > > > > > > > > >>> Hi Andrew, the motivation looks good. A couple of > questions: > > > > > > > > >>> > > > > > > > > >>> FV1: The KIP says frameworks should set them, but it > does not > > > > > > specify > > > > > > > > >>> the mechanism. If these are ordinary user-facing configs, > > > nothing > > > > > > > > >>> prevents an end user from setting arbitrary values, which > > > defeats > > > > > > the > > > > > > > > >>> purpose. Should the framework set them programmatically > > > (builder or > > > > > > > > >>> internal constructor) or is user override intentional? > > > > > > > > >>> > > > > > > > > >>> FV2: What if we have multiple layers of frameworks? Let's > > > say a > > > > > > custom > > > > > > > > >>> framework on top of SpringBoot. I've seen similar > nightmares > > > in the > > > > > > > > >>> past. > > > > > > > > >>> > > > > > > > > >>> Thanks > > > > > > > > >>> Fede > > > > > > > > >>> > > > > > > > > >>> On Tue, Jul 28, 2026 at 8:27 AM Aditya Kousik < > > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > >>>> > > > > > > > > >>>> Hi Andrew, > > > > > > > > >>>> > > > > > > > > >>>> We’re definitely agreed on the increased use of > frameworks > > > over > > > > > > than > > > > > > > > the client directly. I’ve cited the different patterns > Spring, > > > > > > Micronaut, > > > > > > > > smallrye and company-internal frameworks have APIs built on > top > > > of the > > > > > > > > Kafka client, in a couple of KIPs already. > > > > > > > > >>>> > > > > > > > > >>>> The project has enough traction that I think of it as > its > > > own > > > > > > network > > > > > > > > client with distributed log semantics for which frameworks > are > > > written, > > > > > > > > much like gRPC over netty. People want to just write the > business > > > > > > logic and > > > > > > > > leave the plumbing and threading to the frameworks. > > > > > > > > >>>> > > > > > > > > >>>> All of this to say, I’m fine to ship the client > framework > > > as the > > > > > > new > > > > > > > > identifying parameter for frameworks to set. It will be > mighty > > > useful. > > > > > > > > >>>> > > > > > > > > >>>> My addendum, rather than a pushback is that: guilty as > > > charged, > > > > > > I’m > > > > > > > > driven by the otel/DD telemetry/observability of using Apache > > > Kafka in > > > > > > > > applications. The client.framework.version config for > instance > > > can be > > > > > > used > > > > > > > > to detect regressions and isolate root causes. But I feel it > is > > > only > > > > > > one of > > > > > > > > many such facets. You mentioned that you would use client.id > and > > > > > > > > clientInstanceId to identify a client but that these do not > help > > > with > > > > > > > > aggregate/fleet-wide issues. As an example, if I tag an > app’s AZ > > > it can > > > > > > > > help me write alerts on spike in latency in us-west-2. Or, > > > detect stuck > > > > > > > > partitions across multiple client.id/application.names none > of > > > whom > > > > > > share > > > > > > > > the same client.framework.name. > > > > > > > > >>>> > > > > > > > > >>>> Other tags that come to mind: application.name, az, > team, > > > env, > > > > > > rack. > > > > > > > > All fields users usually hijack client.id for. > > > > > > > > >>>> > > > > > > > > >>>> An otel JavaAgent can capture the client metadata > > > registered and > > > > > > > > attach it as tags with each resource span. Users who use > > > frameworks but > > > > > > > > rely on datadog/otel will get visibility into the client > > > metadata for > > > > > > free. > > > > > > > > >>>> > > > > > > > > >>>> The KIP as I read it, serves as a foundation for future > > > use. So I > > > > > > > > don’t want to shoehorn a new behaviour if it explodes the > scope > > > too > > > > > > much. > > > > > > > > >>>> > > > > > > > > >>>> Best regards, > > > > > > > > >>>> Aditya > > > > > > > > >>>> > > > > > > > > >>>>> On Jul 27, 2026, at 13:52, Andrew Schofield < > > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > >>>>> > > > > > > > > >>>>> Hi Aditya, > > > > > > > > >>>>> Thanks for your response. > > > > > > > > >>>>> > > > > > > > > >>>>> AK1: I chose framework as the blessed abstraction > because > > > my > > > > > > focus > > > > > > > > was problem determination for client applications. We often > find > > > that > > > > > > users > > > > > > > > with client problems have not coded directly to the Kafka > client > > > > > > interface > > > > > > > > because they are using a framework. As a result, their > knowledge > > > of the > > > > > > > > application code is one level removed from the Kafka client. > > > > > > Frameworks can > > > > > > > > override configuration defaults, introduce different retry > > > behaviour > > > > > > and so > > > > > > > > on. Lots of companies have their own internal frameworks, so > > > this KIP > > > > > > can > > > > > > > > be used by them too. I'm trying to make it easier to work out > > > when a > > > > > > user > > > > > > > > is making use of a framework and knowing what it is. > > > > > > > > >>>>> > > > > > > > > >>>>> Sometimes, particularly for non-Java clients, people > have > > > > > > overridden > > > > > > > > the ClientSoftwareName/Version themselves, which makes those > > > concepts > > > > > > much > > > > > > > > less useful than they should be. By providing > > > > > > ClientFrameworkName/Version, > > > > > > > > there is no longer any need to do so. That's another > motivation > > > here, > > > > > > even > > > > > > > > though KIPs don't concern themselves with non-Java clients as > > > such. > > > > > > > > >>>>> > > > > > > > > >>>>> We could go for a more flexible key-value approach, > but a > > > simple > > > > > > > > name and version is sufficient for what I had in mind. Feel > free > > > to > > > > > > push > > > > > > > > back with additional justification and examples. > > > > > > > > >>>>> > > > > > > > > >>>>> AK2: Done. o.a.k.clients.CommonClientConfigs. > > > > > > > > >>>>> > > > > > > > > >>>>> AK3: To identify a particular client, I would use > client > > > ID and > > > > > > > > client-instance ID. I think these are generally more useful > > > concepts > > > > > > than > > > > > > > > the framework name and version which are extra information > for > > > the > > > > > > person > > > > > > > > trying to figure out why a client is not behaving as > expected. > > > > > > > > >>>>> > > > > > > > > >>>>> AK4: I'm sure you have more experience of OTel/DataDog > > > > > > collectors. > > > > > > > > You may well be correct that they would be helpful for the > > > collectors. > > > > > > > > >>>>> > > > > > > > > >>>>> Thanks, > > > > > > > > >>>>> Andrew > > > > > > > > >>>>> > > > > > > > > >>>>>> On 2026/07/26 07:48:20 Aditya Kousik wrote: > > > > > > > > >>>>>> Hello Andrew, > > > > > > > > >>>>>> > > > > > > > > >>>>>> I’m reminded of the client.id discussion we had back > in > > > > > > KIP-1313 > > > > > > > > re: client instance id. After that discussion, I have a WIP > KIP > > > that > > > > > > sets a > > > > > > > > foundation for shipping client metadata tags to be sent for > > > telemetry. > > > > > > I > > > > > > > > was hoping we could discuss if part of that approach could > fit > > > this > > > > > > KIP. > > > > > > > > >>>>>> > > > > > > > > >>>>>> AK1. I had a note on the motivation of selecting a > > > “framework” > > > > > > as a > > > > > > > > blessed abstraction. The KIP mentions it’s for the client > > > metadata and > > > > > > > > easier problem diagnosis. This is akin to an “application id” > > > that > > > > > > > > non-framework clients usually tag with. > > > > > > > > >>>>>> > > > > > > > > >>>>>> KIP-606 took an approach like > > > metrics.context.<key>=<val>. If we > > > > > > > > allow client.metadata.<key>=<val>, then a > framework/application > > > name > > > > > > and > > > > > > > > version can sit in such a metadata context and be sent with > > > ApiVersion > > > > > > RPC. > > > > > > > > Of course, this is an open box approach rather than just the > > > framework > > > > > > > > name/version (just two fields) we’re adding to the protocol. > But > > > I’m > > > > > > > > curious about the tier of importance of framework alone. > > > > > > > > >>>>>> > > > > > > > > >>>>>> AK2. Can you clarify if this config goes into > > > > > > > > CommonClientConfigs.java, referenced across > > > producer/consumer/share > > > > > > etc? I > > > > > > > > know some share props have the “share.” prefix going. > > > > > > > > >>>>>> > > > > > > > > >>>>>> AK3. In another thread you mentioned that the broker > may > > > add it > > > > > > to > > > > > > > > the request context. In client logs, clientId is a very > useful > > > string > > > > > > to > > > > > > > > identify WARN logs when things go sideways like broker > > > disconnected, > > > > > > > > rebalance in progress etc. Can you cherry pick and highlight > some > > > > > > useful > > > > > > > > log places that these strings can go? I suppose adding > > > > > > clientId/framework > > > > > > > > to the MDC context might be too voluminous. > > > > > > > > >>>>>> > > > > > > > > >>>>>> AK4. I foresee collectors like Datadog/OTel might find > > > these > > > > > > tags > > > > > > > > useful in each span exported. > > > > > > > > >>>>>> > > > > > > > > >>>>>> Looking forward to your thoughts on this. > > > > > > > > >>>>>> > > > > > > > > >>>>>> Regards, > > > > > > > > >>>>>> Aditya > > > > > > > > >>>>>> > > > > > > > > >>>>>>>> On Jul 17, 2026, at 11:53, Andrew Schofield < > > > > > > > > [email protected]> wrote: > > > > > > > > >>>>>>> > > > > > > > > >>>>>>> Hi, > > > > > > > > >>>>>>> I'd like to open discussion on KIP-1368: Client > > > framework name > > > > > > and > > > > > > > > version. > > > > > > > > >>>>>>> > > > > > > > > >>>>>>> Applications often use application frameworks such as > > > Spring to > > > > > > > > connect to Kafka. To assist with problem diagnosis, this KIP > > > > > > introduces a > > > > > > > > way to provide the framework name and version as part of the > > > metadata > > > > > > the > > > > > > > > client sends to the broker when it connects. > > > > > > > > >>>>>>> > > > > > > > > >>>>>>> Here's the KIP: > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/x/J4Q_Gg__;!!Ayb5sqE7!vs-_AkY76KFOqYK02q4f2tKSkFdnly7eklb5qfewIk841seg2S5cIOXxFhnAixEYsDVuFQyJx8-D5g$ > > > > > > > > >>>>>>> > > > > > > > > >>>>>>> Thanks, > > > > > > > > >>>>>>> Andrew > > > > > > > > >>>>>> > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
