RE: Mirrormaker 2.0 and compacted topics

2020-03-18 Thread Pirow Engelbrecht
.com/Etionlimited> | Instagram<https://www.instagram.com/Etionlimited/> From: Sönke Liebau Sent: Wednesday, 18 March 2020 1:12 PM To: users@kafka.apache.org Subject: Re: Mirrormaker 2.0 and compacted topics Hi Pirow, records at the same offset as in the original topic is not possible for non com

Re: Mirrormaker 2.0 and compacted topics

2020-03-18 Thread Sönke Liebau
Etionlimited> | Instagram > <https://www.instagram.com/Etionlimited/> > > > > *From:* Sönke Liebau > *Sent:* Wednesday, 18 March 2020 12:14 PM > *To:* users@kafka.apache.org > *Subject:* Re: Mirrormaker 2.0 and compacted topics > > > > Hi Pirow, > >

RE: Mirrormaker 2.0 and compacted topics

2020-03-18 Thread Pirow Engelbrecht
.youtube.com/channel/UCUY-5oeACtLk2uTsEjZCU6A> | LinkedIn<https://www.linkedin.com/company/etionltd> | Twitter<https://twitter.com/Etionlimited> | Instagram<https://www.instagram.com/Etionlimited/> From: Sönke Liebau Sent: Wednesday, 18 March 2020 12:14 PM To: users@kaf

Re: Mirrormaker 2.0 and compacted topics

2020-03-18 Thread Sönke Liebau
Hi Pirow, as far as I understand MirrorMaker 2.0 will not treat compacted topics any different from uncompacted topics. What that means for your scenario is that your replication may miss some messages in the case of a long unavailability, if those messages were compacted in the meantime. However

Mirrormaker 2.0 and compacted topics

2020-03-18 Thread Pirow Engelbrecht
Hello, We're currently trying to evaluate Mirrormaker 2.0 for future inter-cluster replication, replacing our bespoke replicator. I understand that Mirrormaker 2.0 documentation is only slated to be released in Kafka 2.5.0, but I was hoping that someone will know whether Mirrormaker 2.0 can be