The issue was due to the fact the some messages were corrupted and didn't get processed at the subscriber end, so nothing to do with Kafka. Since the error logs were not stored anywhere, we had to troubleshoot the live system in debug mode to capture message logs.
On Mon, 21 Oct 2019 at 01:13, M. Manna <manme...@gmail.com> wrote: > Hello, > > I have recently had some message loss for a consumer group under kafka > 2.3.0. > > The client I am using is still in 2.2.0. Here is how the problem can be > reproduced, > > 1) The messages were sent to 4 consumer groups, 3 of them were live and 1 > was down > 2) When the consumer group came back online, the messages were not > received. > > I am using manual offset commit (in a typical while loop-based consumer > thread - same as Kafka Officail Docs), and I only commit offset when I have > fully processed the messages. My Poll time is *4*s. My affected topic has > 6 partitions and 3 replicas. The timeout and connection duration are all > sufficiently set. > > The issue is only when when the consumer group (all consumers) go offline > for a while (e.g. server down) and comes back online. > > Could someone please help me understanding the issue? > > Regards, >
