Hello,
I have setup MM2 and the config is similar to the example config.
https://github.com/apache/kafka/blob/trunk/config/connect-mirror-maker.properties
Primary has 3 brokers and secondary has 1 broker.
I wrote a script to diff number of messages in topics between primary and
secondary cluster.
To get number of messages in topic I use.
bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list
localhost:9092 --topic mytopic --time -1 --offsets 1 | awk -F ":" '{sum +=
$3} END {print sum}'
Is there some reason not all messages gets replicated?
A first thought was that it may have been because I wiped the secondary
cluster and started from beginning. So i switched the name of the secondary
cluster and prefix in mm2 config. But I ended up with the same diff.
Result:
topic: topic1, primary: 236, secondary: 119, diff 117
topic: topic2, primary: 2910, secondary: 1223, diff 1687
topic: topic3, primary: 2248, secondary: 2248, diff 0
topic: topic4, primary: 627, secondary: 91, diff 536
topic: topic5, primary: 84, secondary: 77, diff 7
topic: topic6, primary: 333, secondary: 156, diff 177
topic: topic7, primary: 158, secondary: 45, diff 113
topic: topic8, primary: 247, secondary: 48, diff 199
...
...
BR
Nils
