Roger, My understanding of both, beyond what Zookeeper already does, are:
1. Consul can be used to monitor a service and report it's status. This can be very useful for knowing if a service, such as Zookeeper of Kafka, goes down. This can be done through a built-in web interface. 2. Confd leverages Consul, or etcd, to propogate changes to a service and restart it if necessary. So, if we change a broker specific setting, we can put the change in Consul and have Confd automatically modify the config files on the broker nodes and restart the service as needed. My knowledge in this area is a bit limited as I haven't used either. I'm working with someone who is and wanted to ask people about this so that we can learn what works and what doesn't. _______________________________________ From: Roger Hoover [roger.hoo...@gmail.com] Sent: Friday, October 17, 2014 12:26 PM To: users@kafka.apache.org Subject: Re: Kafka/Zookeeper deployment Questions Casey, Could you describe a little more about how these would help manage a cluster? My understanding is that Consul provides service discovery and leader election. Kafka already uses ZooKeeper for brokers to discover each other and elect partition leaders. Kafka high-level consumers use ZK to divide up topic partitions amongst themselves. I'm not able to see how Consul +/or confd will help. Cheers, Roger
