Hi all, I updated the KIP and the draft PoC after investigating the broader set of classes covered by KIP-1265.
In my earlier replies, I proposed a general transition policy and used several classes as examples. The full scan showed that this scope would be difficult to apply consistently: - The scan found a large number of non-`@Public` candidates across published Kafka modules, with downstream usage spread across many classes. - The candidates are not one uniform group. Some appear to be real public APIs that are missing `@Public` and should be reviewed separately. Others are internal implementation classes exposed through public packages. - Deprecating the candidates one by one would require a separate decision about each class's intended audience, downstream usage, replacement guidance, and removal timing. For candidates confirmed to be internal implementation details, adding individual compatibility shims would spend substantial maintainer effort preserving an accidental API surface that Kafka does not intend applications to use or Kafka to support. That cost is not justified for every internal class. Compiler warnings in Kafka's own code could be suppressed, but that would not resolve these class-by-class API decisions. Representative exact-import searches include [`Utils`](https://github.com/search?q=%22import+org.apache.kafka.common.utils.Utils%3B%22+-repo%3Aapache%2Fkafka&type=code), [`Time`](https://github.com/search?q=%22import+org.apache.kafka.common.utils.Time%3B%22+-repo%3Aapache%2Fkafka&type=code), [`CommonClientConfigs`](https://github.com/search?q=%22import+org.apache.kafka.clients.CommonClientConfigs%3B%22+-repo%3Aapache%2Fkafka&type=code), [`OffsetResetStrategy`](https://github.com/search?q=%22import+org.apache.kafka.clients.consumer.OffsetResetStrategy%3B%22+-repo%3Aapache%2Fkafka&type=code), and [`Cache`](https://github.com/search?q=%22import+org.apache.kafka.common.cache.Cache%3B%22+-repo%3Aapache%2Fkafka&type=code). These searches are examples of the scale, not a complete census of downstream usage. Based on this investigation, I narrowed KIP-1320 as follows: 1. The implementation in this KIP now changes only `org.apache.kafka.common.utils.Utils`. 2. KIP-1320 does not create a blanket compatibility guarantee for every class that KIP-1265 classifies as Private. 3. `Utils` receives an explicit staged transition because it has the highest observed import count: Kafka 4.4 keeps the original class as a deprecated compatibility wrapper, and Kafka 5.0.0 removes that wrapper. 4. The earlier ByteBuffer stream and `ProducerIdAndEpoch` examples are removed from the KIP and PoC. 5. Other Private classes remain governed by KIP-1265 and do not automatically receive a compatibility shim. The KIP also proposes that Kafka maintainers may reuse the same transition pattern for another Private class when downstream usage evidence or compatibility feedback shows that a direct change would have significant impact. Such a change would still require a Jira, normal code review, tests, an upgrade-note entry, and an explicit major-release removal boundary, but it would not require another KIP solely to approve the already-defined shim pattern. This remains a case-by-case decision and is not a blanket compatibility promise. KIP-1320 is still useful after KIP-1265 for the `Utils` case because the Private annotation does not produce a standard Java compiler warning, the downstream checker is opt-in, and KIP-1265 does not define a class-specific removal schedule. I would appreciate feedback on the narrowed `Utils`-only implementation and on allowing the same transition pattern to be reused case by case. Thanks, Eric On 2026/07/04 12:08:11 Eric Chang wrote: > Hi Chia, > > Good question -- working through "why is this still needed after KIP-1265" > actually made me want to reshape the KIP, so let me give the reasoning and > then two things I'd like your take on. > > Why it's still needed: KIP-1265 classifies audience and detects usage, but > @InterfaceAudience.Private produces no compiler warning (only > java.lang.Deprecated does), its checker only runs if the consumer adds it to > their own build, and it sets no timeline. So for classes that are already > exposed and used, KIP-1265 alone leaves third parties with no signal and no > cutoff. KIP-1320 fills that: a zero-setup @Deprecated compile-time warning > during 4.x, plus a concrete "no compatibility guarantee from 5.0.0" policy. > > That reframing makes the KIP feel more like a *policy* than a single-class > change, so two questions: > > 1. Retitle? From "Deprecate org.apache.kafka.common.utils.Utils and > introduce an internal replacement" to something like "Deprecation and > removal policy for internal classes exposed in public packages", to > reflect that it's now primarily a policy. > > 2. Approach? Make the KIP's core a single policy -- "@Private-but-in-a- > public-package classes are unsupported from 5.0.0" -- that covers the > whole surface with no per-class code, and keep only a small curated set > (the current Utils / ByteBufferInputStream / ByteBufferOutputStream) as > the reference @Deprecated example. (Most deprecations wouldn't even need > an internals move -- classes with only a few internal callers can just be > @Deprecated in place; only heavy-internal-use ones like Utils get an > internals replacement.) Further deprecations would then be follow-up > JIRAs under the policy rather than new KIPs. > > If that direction sounds reasonable I'll write it up in full (scope > criteria, class buckets, external-usage data). WDYT? > > Thanks, > Eric > > On 2026/07/03 21:27:47 Chia-Ping Tsai wrote: > > hi Eric > > > > Would you mind including KIP-1265 in this KIP? It would be cool to > > elaborate on why we still need this KIP even though KIP-1265 has already > > been merged > > > > Best, > > Chia-Ping > > > > On 2026/04/15 23:53:50 Eric Chang wrote: > > > Hi Kafka devs, > > > > > > I would like to start the discussion for KIP-1320: > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1320%3A+Deprecate+org.apache.kafka.common.utils.Utils+and+introduce+an+internal+replacement > > > > > > This KIP proposes to deprecate `org.apache.kafka.common.utils.Utils` > > > in Kafka `4.4.0`, introduce > > > `org.apache.kafka.common.utils.internals.Utils` for Kafka internal > > > usage, and remove the old `org.apache.kafka.common.utils.Utils` class > > > in Kafka `5.0.0`. > > > > > > The proposed migration keeps `org.apache.kafka.common.utils.Utils` > > > source and binary compatible during the Kafka `4.4.x` line. The > > > internal replacement is created by copying the existing > > > implementation, so this KIP does not propose changing the behavior of > > > any `Utils` method. > > > > > > Please take a look and let me know your thoughts. > > > > > > Thanks, > > > Eric Chang > > > > > >
