Hello,
I had spent a while trying to figure out why I cannot see
«kafka.connect.mirror:type=MirrorCheckpointConnector» Mbean and, since I've got
stuck, I'm seeking assistance from the community.
My setup:
* MM2 is running inside Docker container (AWS ECS Fargate if that might
matter)
* Src and Dst Kafka custers are AWS MSK (if this might matter)
* Base Docker image: public.ecr.aws/bitnami/kafka:latest (INFO Kafka
version: 3.4.0)
* ENV JMX_PORT 9010
*
CMD ["/opt/mirrormaker/run.sh"] (run.sh is pretty simple and can be described
as /opt/bitnami/kafka/bin/connect-mirror-maker.sh
"/opt/mirrormaker/mm2.properties")
mm2.properties file, that is copied into container, with sensitive data
replaced is below:
clusters = devsdpuse1msktestsourc, devsdpuse1msktestdest
testsrc.bootstrap.servers = b-3.src[…]:9092,b-2.src[…]:9092,b-1.src[…]:9092
testdst.bootstrap.servers = b-2.dst[…]:9092,b-1.dst[…]:9092,b-3.dst[…]:9092
testsrc->testdst.enabled = true
testsrc->testdst.topics = msk
testsrc->testdst.groups = .*
replication.factor = 3
sync.topic.acls.enabled = false
tasks.max = 1
# Avoid prefixing topics on destination with source cluster name
replication.policy.separator =
source.cluster.alias =
target.cluster.alias =
checkpoints.topic.replication.factor=3
heartbeats.topic.replication.factor=3
offset-syncs.topic.replication.factor=3
offset.storage.replication.factor=3
status.storage.replication.factor=3
config.storage.replication.factor=3
Replication works just fine, though I see only MirrorSourceConnector MBean
while MirrorCheckpointConnector MBean is missing:
root@ip-xx-xx-xx-xx:/opt/mirrormaker# ./jmx-dump -p 9010 --dump-all | jq
'with_entries(select(.key | startswith("kafka.connect.mirror:")))'
{
"kafka.connect.mirror:partition=0,topic=msk,type=MirrorSourceConnector": {
"record-count": 2,
"record-age-ms-max": "NaN",
"record-rate": 0,
"record-age-ms": 21,
"replication-latency-ms-avg": "NaN",
"byte-count": 56,
"replication-latency-ms-min": "NaN",
"record-age-ms-avg": "NaN",
"record-age-ms-min": "NaN",
"byte-rate": 0,
"replication-latency-ms-max": "NaN",
"replication-latency-ms": 24
}
}
Side notice: also as you can see MirrorSourceConnector has only "topic" and
"partition", despite docu describes "target" as well:
https://github.com/apache/kafka/blob/trunk/connect/mirror/README.md#monitoring-an-mm2-process
The main Q is: where is the "MirrorCheckpointConnector" Mbean and why it is
missing?
Please let me know if any additional info may be needed.
Also since this is a test setup I can change/adjust it if needed.
Thanks in advance.
--
Georgii Iermulnik
