Hi, We are using mirrormaker to mirror topics from one cluster to another, and I wanted to get some advice from the community on how people are doing mirroring. In particular, how are people dealing with topic creation?
Do you turn on auto-topic creation in your destination clusters (auto.create.topics.enable=true)? If not, do you manually create the individual destination topics? If so, how does that work with mirroring based on a whitelist (regex)? The way we are doing it right now is, we have our regex in a file somewhere. The regex is used in 2 ways: 1) Passed to mirrormaker, to do the mirroring. 2) Passed to a program which looks up all the topics on the source cluster, finds the ones that match the regex, and then creates them on the destination cluster. (We have auto-topic creation turned off auto.create.topics.enable=false) One downside of that approach is there a potential race, where if the regex changes, then mirrormaker (in #1) might start trying to produce to a new destination topic before the topic was created (by #2). Some other hand-wavy ideas that came to mind might be: * handling topic creation in a MirrorMakerMessageHandler * handling topic creation in an interceptor Anyway, was hoping to get some thoughts from people who are already doing this. Thanks! -James
