Hi Mark, For global Pub/Sub between clusters, I think you might need another layer of service to direct users to the right Kafka cluster. Yes, mirror maker could be used for cross colo replication. Currently mirror maker cannot be used for bi-direction mirror yet, after KAFKA-1997 get checked in, you can use the message handler to do filtering so you will not have the infinite loop.
Jiangjie (Becket) Qin From: Mark Flores <mark.flo...@expeditors.com<mailto:mark.flo...@expeditors.com>> Reply-To: "users@kafka.apache.org<mailto:users@kafka.apache.org>" <users@kafka.apache.org<mailto:users@kafka.apache.org>> Date: Monday, March 9, 2015 at 1:04 PM To: "users@kafka.apache.org<mailto:users@kafka.apache.org>" <users@kafka.apache.org<mailto:users@kafka.apache.org>> Subject: Kafka Questions One of our development teams is considering implementing a Kafka solution. If the development team were to assume implementing 6 separate regional Kafka clusters: · How could we implement global Pub/Sub between clusters? · Can we do bi-directional replication with MirrorMaker between each cluster? · How do we prevent a message published at one cluster being replicated to another cluster and into an infinite loop (assuming we use a MirrorMaker whitelist like Global.* at each cluster)? Regards, Mark Flores Project Manager, Enterprise Technology Direct 206-576-2675 Email mark.flo...@expeditors.com<mailto:mark.flo...@expeditors.com> [expd] Global Headquarters, Seattle 1015 Third Avenue, 12th Floor Seattle, WA 98104
