I have some doubts regarding message consumption and production, as well as transactional capabilities. I am using a Kafka template to produce a message within a transaction. After that, I execute another transaction that produces a message and intentionally throws a runtime exception to simulate a transaction rollback.
Next, I use the Kafka AdminClient to retrieve the latest offset for the topic partition and the consumer group's offsets for the same topic partition. However, when I compare the offset numbers, I notice a difference. In this example, the consumer has 4 offsets, while the topic has only 2. I have come across references to this issue in a Spring-Kafka report, specifically in the Kafka-10683 report, where developers describe it as either Bogus or Pseudo Lag. I am keen on resolving this problem, and I would greatly appreciate hearing about your experiences and knowledge regarding this matter. Thank you very much Henry