Hi, thanks for the reply! See comments below - On Mon, Jun 5, 2023 at 9:59 AM Richard Bosch <richard.bo...@axual.com> wrote:
> This message indicates that the transactional producer uses the send > offsets to transactions option. > We use, by default, transactional producers (setting a transaction.id). And most services are setup to also use transactional consumers (read_committed with auto-commit). This seems to work fine as a general pattern - and it's only for these two consumers we see this error. > It means that either another producer application is writing offsets > for the consumer group as well or that you might be having a consumer > and producer that adds offset to the transaction in the same > application. > > I was under the impression that producers store offsets in a the __transaction_state topic, and that consumers store offsets in __consumer_offsets. And that transactional markers are written to the target partition by the transactional producer, and that the broker uses this to dish out messages to consumers (transactional or non-transactional) in a more or less transparent way. How would the producer need to be configured in order for it to "mess with" the consumer group id used by the consumer in this case? 🤔 Now, looking closer at the services that uses these consumer group ids, they both use `spring-kafka` (as do all other services) with an error handler setup (`org.springframework.kafka.listener.DefaultErrorHandler`). This handler seems to be doing seeks on the consumer, and I'm thinking this might be what's somehow causing this... I'll try to dig deeper into that error handler -- but if anyone have experience from Spring-Kafka and error handlers, feel free to jump in :) Cheers /Anders > > On Mon, Jun 5, 2023 at 8:49 AM Anders Engström <epirea...@gmail.com> > wrote: > > > > Hi! > > > > We're running Kafka 3.3 (at Aiven) and are seeing warning logs about > offset > > commits from both consumers and producers. > > This happens for two of our consumers (out of hundreds). I've tried > > Googling for the error (of course) but I haven't found anything obvious. > > > > [kafka-01-78]2023-06-05T06:44:12.536334[kafka][2023-06-05 06:44:12,536] > > > WARN [GroupMetadataManager brokerId=78] group: <group.id> with leader: > > > <consumer-group-leader-id> has received offset commits from consumers > as > > > well as transactional producers. Mixing both types of offset commits > will > > > generally result in surprises and should be avoided. > > > (kafka.coordinator.group.GroupMetadataManager:70) > > > > > > Any recommendations on what to look for in the application > > consumer/producer code? The applications using the consumer group id is > not > > really special compared to all other consumer applications. > > > > Best Regards /Anders >
