Hi Ruben,
The offset-syncs topic is necessary when the checkpointing feature of
MM2 is being used (a.k.a. consumer group offset replication).
Since the offsets of the original and the replica topic are not
guaranteed to exactly match, there has to be a translation step when
replicating the committed group offsets. The offset-syncs topic is used
for storing these "mappings".
The data replication (MirrorSourceConnector & Task) is responsible for
writing into this topic.
I don't think there is a way to disable this feature. If you don't use
checkpointing, you can safely ignore the issue.
Alternatively, I remember that there was a feature around configuring
MM2 to put the offset-syncs topic into the target cluster instead of the
source cluster.
Daniel
2022.01.18. 11:01 keltezéssel, Ruben de Vries írta:
We've been trying to setup a MirrorMaker 2 to mirror data from a cluster on
which we only have read access.
It seems to work, but the logs are hammered with
TopicAuthorizationException errors in the MirrorCheckpointConnector thread
for a mm2-offset-syncs.target-cluster.internal topic.
At this point I think I'm fairly certain it's trying to access this topic
on the source cluster, which is possible in our case ...
I've been trying to dig through the codebase to figure out what the purpose
and importance is of this topic or if it can be disabled, but that's a bit
too much of a challenge haha.
So I'm coming here to ask:
- Could we somehow disable this?
- Or could we safely ignore this?
Ruben
--
Az e-mailen az AVG vírusellenőrzést végzett.
http://www.avg.com
