Thanos Athanasopoulos created KAFKA-16798: ---------------------------------------------
Summary: Mirrormaker2 consumer groups sync.group.offsets.interval not working Key: KAFKA-16798 URL: https://issues.apache.org/jira/browse/KAFKA-16798 Project: Kafka Issue Type: Bug Components: mirrormaker Affects Versions: 3.7.0 Reporter: Thanos Athanasopoulos Single instance MirrorMaker2 in dedicated mode, active passive replication logic. sync.group.offsets.interval.seconds=2 configuration is enabled and active {noformat} [root@xxxxx-xxxxxxxxx ~]# docker logs cc-mm 2>&1 -f | grep -i "auto.commit.interval\|checkpoint.interval\|consumer.commit.interval\|sync.topics.interval\|sync.group.offsets.interval\|offset-syncs.interval.seconds " auto.commit.interval.ms = 5000 auto.commit.interval.ms = 5000 auto.commit.interval.ms = 5000 auto.commit.interval.ms = 5000 auto.commit.interval.ms = 5000 auto.commit.interval.ms = 5000 sync.group.offsets.interval.seconds = 2 sync.group.offsets.interval.seconds = 2 auto.commit.interval.ms = 5000 sync.group.offsets.interval.seconds = 2 auto.commit.interval.ms = 5000 auto.commit.interval.ms = 5000 auto.commit.interval.ms = 5000 auto.commit.interval.ms = 5000 auto.commit.interval.ms = 5000 auto.commit.interval.ms = 5000 auto.commit.interval.ms = 5000 auto.commit.interval.ms = 5000 sync.group.offsets.interval.seconds = 2 auto.commit.interval.ms = 5000 auto.commit.interval.ms = 5000 sync.group.offsets.interval.seconds = 2 sync.group.offsets.interval.seconds = 2 auto.commit.interval.ms = 5000 {noformat} but is not working, the commit of offsets happens always in 60 seconds as you can see in the logs {noformat} [2024-05-20 09:32:44,847] INFO [MirrorSourceConnector|task-0|offsets] WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:32:44,852] INFO [MirrorSourceConnector|task-1|offsets] WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:32:44,875] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:32:44,881] INFO [MirrorCheckpointConnector|task-0|offsets] WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:32:44,890] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:33:44,850] INFO [MirrorSourceConnector|task-0|offsets] WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 21 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:33:44,854] INFO [MirrorSourceConnector|task-1|offsets] WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:33:44,878] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:33:44,883] INFO [MirrorCheckpointConnector|task-0|offsets] WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 2 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:33:44,895] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:34:44,852] INFO [MirrorSourceConnector|task-0|offsets] WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:34:44,857] INFO [MirrorSourceConnector|task-1|offsets] WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:34:44,880] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:34:44,886] INFO [MirrorCheckpointConnector|task-0|offsets] WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:34:44,897] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:35:44,855] INFO [MirrorSourceConnector|task-0|offsets] WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:35:44,860] INFO [MirrorSourceConnector|task-1|offsets] WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:35:44,883] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:35:44,888] INFO [MirrorCheckpointConnector|task-0|offsets] WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:35:44,901] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:36:44,860] INFO [MirrorSourceConnector|task-0|offsets] WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:36:44,863] INFO [MirrorSourceConnector|task-1|offsets] WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:36:44,886] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:36:44,893] INFO [MirrorCheckpointConnector|task-0|offsets] WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:36:44,904] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:37:44,863] INFO [MirrorSourceConnector|task-0|offsets] WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 21 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:37:44,866] INFO [MirrorSourceConnector|task-1|offsets] WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:37:44,888] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:37:44,895] INFO [MirrorCheckpointConnector|task-0|offsets] WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:37:44,906] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:38:44,866] INFO [MirrorSourceConnector|task-0|offsets] WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:38:44,869] INFO [MirrorSourceConnector|task-1|offsets] WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:38:44,890] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:38:44,898] INFO [MirrorCheckpointConnector|task-0|offsets] WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 2 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:38:44,910] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:39:44,869] INFO [MirrorSourceConnector|task-0|offsets] WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:39:44,872] INFO [MirrorSourceConnector|task-1|offsets] WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:39:44,892] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:39:44,900] INFO [MirrorCheckpointConnector|task-0|offsets] WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) [2024-05-20 09:39:44,915] INFO [MirrorHeartbeatConnector|task-0|offsets] WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233) {noformat} MM2 Configuration: {noformat} clusters = cl01,cl02 cl01.bootstrap.servers = cl01-kafka-01:9092,cl01-kafka-02:9092,cl01-kafka-03:9092 cl02.bootstrap.servers = cl02-kafka-01:9092,cl02-kafka-02:9092,cl02-kafka-03:9092 ############################## ### First cluster config ############################## # source.cluster.alias = cl01 # target.cluster.alias = cl02 cl01->cl02.enabled=true cl01->cl02.enable.auto.commit=true cl01->cl02.sync.group.offsets.enabled=true cl01->cl02.refresh.groups.enabled=true cl01->cl02.refresh.topics.enabled=true cl01->cl02.sync.topics.enabled=true cl01->cl02.sync.topic.acls.enabled=false cl01->cl02.sync.acls.enabled=false cl01->cl02.emit.heartbeats.enabled=true cl01->cl02.emit.checkpoints.enabled=true cl01->cl02.sync.topic.configs.enabled=true cl01->cl02.emit.offset-syncs.enabled=true cl01.consumer.auto.offset.reset=earliest ############################## ### Second cluster (DR) config ############################## # source.cluster.alias = cl01 # target.cluster.alias = cl02 cl02->cl01.enabled=false cl02->cl01.enable.auto.commit=false cl02->cl01.emit.offset-syncs.enabled=false cl02->cl01.sync.group.offsets.enabled=false cl02->cl01.sync.topic.acls.enabled=false cl02->cl01.sync.acls.enabled=false cl02->cl01.refresh.groups.enabled=true cl02->cl01.refresh.topics.enabled=true cl02->cl01.sync.topics.enabled=true cl02->cl01.emit.heartbeats.enabled=true cl02->cl01.emit.checkpoints.enabled=true cl02->cl01.sync.topic.configs.enabled=true #cl02.consumer.auto.offset.reset=earliest ############################## ### CC Configs ############################## group.id=crosscluster-consumer-group client.id=crosscluster-clientgroups=.* #groups='.*cc-consumer-group.*' offset-syncs.topic.whitelist='cl.*\.test-topic' #topics='cl.*\.test-topic' topics.blacklist='[.*[\-\.]internal, .*\.replica, __.*]' key.converter = org.apache.kafka.connect.converters.ByteArrayConverter value.converter = org.apache.kafka.connect.converters.ByteArrayConverteremit.heartbeats.interval.seconds=5 emit.checkpoints.interval.seconds=5 emit.offset-syncs.interval.seconds=2 auto.commit.interval.seconds=5 checkpoint.interval.seconds=5 consumer.commit.interval.seconds=5 offset.flush.interval.seconds=5 sync.topics.interval.seconds=2 sync.group.offsets.interval.seconds=2 sync.acls.interval.seconds=5 refresh.groups.interval.seconds=5 refresh.topics.interval.seconds=5 max.poll.interval.seconds=5 heartbeats.topic.retention.hours=1 checkpoints.topic.retention.hours=1 offset.syncs.topic.retention.hours=1 errors.log.enable=true errors.log.include.messages=true dedicated.mode.enable.internal.rest=true cluster.producer.enable.idempotence=true allow.auto.create.topics=truetasks.max=48 replication.factor=1 config.storage.replication.factor=1 offset.storage.replication.factor=1 status.storage.replication.factor=1 offset-syncs.topic.replication.factor=3 heartbeats.topic.replication.factor=1 checkpoints.topic.replication.factor=1 offsets.storage.replication.factor=1{noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)