Hi Everyone, since kafka 0.11.x supports exactly-once semantics, I want to be sure, that it is possible to achieve it across kafka clusters using MirrorMaker.
We have got two locations with "primary" cluster in each location and for each location we have got one "aggregation" cluster which mirrors data from all primary clusters. Currently we deduplicate messages when we copying data from aggregation kafka to HDFS by separete YARN application. But in aggregation kafka duplicates remains. So I want to ensure that there are no duplicates and loss data in kafka and not to use our deduplication yarn application anymore. If it is possible, how to configure MirrorMaker to achieve exactly-once delivery across primary and aggregation clusters? Thanks and have a nice day, Jiri Humpolicek Je dobré vědět, že tento e-mail a přílohy jsou důvěrné. Pokud spolu jednáme o uzavření obchodu, vyhrazujeme si právo naše jednání kdykoli ukončit. Pro fanoušky právní mluvy - vylučujeme tím ustanovení občanského zákoníku o předsmluvní odpovědnosti. Pravidla o tom, kdo u nás a jak vystupuje za společnost a kdo může co a jak podepsat naleznete zde<https://onas.seznam.cz/cz/podpisovy-rad-cz.html> You should know that this e-mail and its attachments are confidential. If we are negotiating on the conclusion of a transaction, we reserve the right to terminate the negotiations at any time. For fans of legalese—we hereby exclude the provisions of the Civil Code on pre-contractual liability. The rules about who and how may act for the company and what are the signing procedures can be found here<https://onas.seznam.cz/cz/signature-rules.html>.
