Re: Mirror maker setup - multi node

2016-06-28 Thread Gerard Klijs
Then it's much simpler, just use replication-factor=3 for your topics, either by setting it when creating topics manually, or by setting default.replication.factor=3 in the broker settings. Kafka will then take care of the leadership on the partitions so both the load is balanced, and in case some

Re: Mirror maker setup - multi node

2016-06-28 Thread cs user
Hi there, I mean 1 cluster with 3 nodes. So I will need to run the mirror maker cluster on each of the 3 nodes in the cluster, in case of the loss of a node, the other 2 will continue to pull messages off the consumer cluster. It does seem to work correctly when I tested it. It just warns about to

Re: Mirror maker setup - multi node

2016-06-28 Thread Gerard Klijs
With 3 nodes, I assume you mean 3 clusters? If I understand correctly, say you have 3 clusters, A, B, and C, you simultaneously: - want to copy from A and B to C, to get an aggregation in C - want to copy fram A and C to B, to get a fail-back aggregation in B. Now what will happen when a message is