We migrated data from 7 node source cluster to target 9 node cluster. MM source connector producer does not have any compression as i mentioned earlier uses default producer. We use ByteArrayConverter for both producer key and value, idempotency =true. Is there a chance the JMX metrics can go wrong ?.
On Thu, Apr 29, 2021 at 12:09 AM fighter <kumarg.pa...@gmail.com> wrote: > We have did the kafka cluster migration from source kafka cluster to > target kafka cluster using MirrorMaker 2.5.1 in distributed mode using > kafka connect cluster. We see noticeable difference incoming messages rate > per sec on source and target. We also analyze that on kafka connect > producer has less output rate compared to consuming rate of > consumer.Details are given below. Do you suspect the message loss from > kafka connect worker ?. We have seen the offsetsync topic the snapshots and > data rate is quite well. Any help will be appreciated. > > Kafka connect Consumer enter image description here > <https://i.stack.imgur.com/YQGga.png>. kafka connect Producer enter image > description here <https://i.stack.imgur.com/asp4t.png> > > source kafka:enter image description here > <https://i.stack.imgur.com/JrKmK.png> target kafka: enter image > description here <https://i.stack.imgur.com/Umtx7.png> > > We notices Mirror maker tasks and workers has no issues. We have three > connect worker nodes only one connect node has this producer & consumer > difference. As we don't have logs any insight will help to investigate the > issue. Kafka MM Source connector producer and consumer uses default config > only. > > >
