> In your example, us-west.us-east.us-central.us-west.topic is an invalid > "remote topic" name because us-west appears twice. MM2 will not replicate > us-east.us-central.us-west.topic into us-west a second time, because the > source topic already has us-west in the prefix. This is what I mean by > "cycle detection" -- cyclical replication does not result in infinite > recursion.
Oh - got it, it checks the entire prefix, which seems obvious to me in retrospect :) Rhys > On Oct 15, 2018, at 3:18 PM, Ryanne Dolan <ryannedo...@gmail.com> wrote: > > Rhys, thanks for your enthusiasm! > > In your example, us-west.us-east.us-central.us-west.topic is an invalid > "remote topic" name because us-west appears twice. MM2 will not replicate > us-east.us-central.us-west.topic into us-west a second time, because the > source topic already has us-west in the prefix. This is what I mean by > "cycle detection" -- cyclical replication does not result in infinite > recursion. > > It's important to note that MM2 does NOT disallow these sort of cycles, it > just knows how to deal with them properly. > > Also notice this is done at the topic level, not per record. The records > don't need any special header or anything for this cycle detection > mechanism to work. > > Thanks! > Ryanne > > On Mon, Oct 15, 2018 at 3:40 PM McCaig, Rhys <rhys_mcc...@comcast.com> > wrote: > >> Hi Ryanne, >> >> This KIP is fantastic. It provides a great vision for how MirrorMaker >> should evolve in the Kafka project. >> >> I have a question on cycle detection - In a scenario where I have 3 >> clusters replicating between each other, it seems it may be easy to >> misconfigure the connectors if auto topic creation is turned on so that >> records become replicated to increasingly longer topic names (until the >> topic name limit is reached). Consider clusters us-west, us-central, >> us-east: >> >> us-west: topic >> us-central: us-west.topic >> us-east: us-central.us-west.topic >> us-west: us-east.us-central.us-west.topic >> us-central: us-west.us-east.us-central.us-west.topic >> >> I’m not sure whether this scenario would actually justify implementing >> additional measures to avoid such a configuration, rather than ensuring >> that the documentation is clear on how to avoid such scenarios - would be >> good to hear what others think on this. >> >> Excited to see the discussion on this one. >> >> Rhys >> >>> On Oct 15, 2018, at 9:16 AM, Ryanne Dolan <ryannedo...@gmail.com> wrote: >>> >>> Hey y'all! >>> >>> Please take a look at KIP-382: >>> >>> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0 >>> >>> Thanks for your feedback and support. >>> >>> Ryanne >> >>
