Hi everyone, Gentle nudge on this. Motivation is still a driving factor for this KIP. Spring, smallrye, micronaut (and to an extent ShareConsumers with an ack) all implement a per-record ack.
It would be good to support a 1P approach at the client level so users can add hooks around the scope of consuming a single record (rather than just the batch as is with the current consumer interceptor). This helps cut boilerplate around instrumentation, mdc context setup etc. Cheers, Aditya > On May 17, 2026, at 14:58, Aditya Kousik <[email protected]> wrote: > > Hi all, > > Adding on a couple more points for motivation: > > 1. Users typically set MDC context from each record but this can break in > subtle ways for both sync/async threading models > > for (var r: consumer.poll(…)) { > MDC.put(“kafka.offset”, r.offset()); > // add rest of the fields > process(r); // throws an ex > MDC.clear(); > } > > In both sync and async models, record N+1 carries prev record’s context > during exceptions. The solution is a try finally for cleanup, yet more > boilerplate for teams to maintain manually. Across thread boundaries, we have > to manually copy threadlocals around from the poller loop (such as OTel,DD > spans). > > In KafkaShareConsumer, this behaviour is more common when worker threads can > perform individual redelivery, ack calls. Sometimes, a main poll loop for > try/catch does not neatly cleanup state between records. > > 2. I also see Confluent’s parallel-consumer implement a PollContext to > achieve a similar function to wrap each poll() action per-record. > > Best, > Aditya > >> On Apr 16, 2026, at 11:45, Aditya Kousik <[email protected]> wrote: >> >> Hi kafka dev, >> >> Gentle ping on this KIP. I’ve added a short [unit test >> reproducer](https://github.com/apache/kafka/pull/22077) that highlights the >> gap mentioned in the KIP. >> >> Would appreciate any eyes on both. >> >> Thanks, >> Aditya >> >>>> On Apr 1, 2026, at 08:14, Aditya Kousik <[email protected]> wrote: >>> >>> Hi all, >>> >>> I'd like to start a discussion on KIP-1305: Per-Record Context Processing >>> for Kafka Consumers. >>> >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1305%3A+Per-Record+Context+Processing+for+Kafka+Consumers >>> >>> Spring Kafka, SmallRye, Micronaut, OTel, and Datadog have each >>> independently built per-record lifecycle hooks because the client doesn't >>> provide one. This KIP proposes ConsumerRecordContextProcessor, a >>> configurable hook that creates a per-record AutoCloseable context via >>> record.newContext(), giving frameworks and instrumentation libraries a >>> single stable integration point. >>> >>> Looking forward to your feedback. >>> >>> Thanks, >>> Aditya Kousik >
