Re: kafka MirrorMaker2 - stop offset sync when consumer group idle on source cluster

2024-10-04 Thread Mehrtens, M
You may be running into the issue described here: https://issues.apache.org/jira/browse/KAFKA-16364 If that's the case, I believe you have two options: 1. Reduce emit.checkpoints.interval.seconds and sync.group.offsets.interval.seconds to reduce the offset lag between source and target cluster

Re: kafka MirrorMaker2 - stop offset sync when consumer group idle on source cluster

2024-10-04 Thread Greg Harris
Hi Massimiliano, A small amount of re-delivery is expected when using MirrorMaker2 3.5+ offsets replication. This is because MM2 does not offer exactly once offset replication. I gave a conference talk recently about this which you may find informative: https://current.confluent.io/2024-sessions/