We have did the kafka cluster migration from source kafka cluster to target
kafka cluster using MirrorMaker 2.5.1 in distributed mode using kafka
connect cluster. We see noticeable difference incoming messages rate per
sec on source and target. We also analyze that on kafka connect producer
has less output rate compared to consuming rate of consumer.Details are
given below. Do you suspect the message loss from kafka connect worker ?.
We have seen the offsetsync topic the snapshots and data rate is quite
well. Any help will be appreciated.

 Kafka connect Consumer enter image description here
<https://i.stack.imgur.com/YQGga.png>. kafka connect Producer enter image
description here <https://i.stack.imgur.com/asp4t.png>

source kafka:enter image description here
<https://i.stack.imgur.com/JrKmK.png> target kafka: enter image description
here <https://i.stack.imgur.com/Umtx7.png>

We notices Mirror maker tasks and workers has no issues. We have three
connect worker nodes  only one connect node has this producer & consumer
difference. As we don't have logs any insight will help to investigate the
issue. Kafka MM Source connector producer and consumer uses default config
only.

Reply via email to