Using https://github.com/apache/kafka/tree/trunk/connect/mirror as a guide,
I have build from source the origin/KIP-382 of
https://github.com/apache/kafka.git.

I am seeing 2 issues

* I brought up 2 processes on 2 different nodes ( they are actually pods on
k8s but that should not matter ). They share the mm2.properties file and
are replicating ( 1-way ) 3 topics with 8 partitions in total.  That seems
to be the way to create a standalone mm2 cluster. I do not however see( at
least the mbeans do not show ) any attempt to rebalance.
https://github.com/apache/kafka/tree/trunk/connect/mirror#monitoring-an-mm2-process
mbeans
are all on a single node



* I restart the processes on the 2 nodes ( hard stop ans start ). The
offsets for replication seem to be reset to the earliest, as if it is a
brand new mirroring. It is also obvious from the
"record-age-ms-avg|replication-latency-ms-avg"
which I track through the restart.




This implies that


1. Load balancing by rebalancing is not working. I cannot scale up or down
by adding nodes to the mm2 cluster or removing them.

2. Restore on a mirror is not working. If the MM2 cluster is brought down,
it does not start mirroring from the last known state. I see the,
state/config topics etc created as expected..





The mm2.properties is pretty mimimal


*clusters = a , b*



*a.bootstrap.servers = k.....*

*b.bootstrap.servers = k.....*


*# only allow replication dr1 -> dr2*

*a->b.enabled = true*

*a->b.topics = act_search_page*

*a->b.emit.heartbeats.enabled = false*


*b->a..enabled = false*

*b->a.emit.heartbeats.enabled = false*






What do you think is the issue ?


Thanks

Reply via email to