Casey, Could you describe a little more about how these would help manage a cluster?
My understanding is that Consul provides service discovery and leader election. Kafka already uses ZooKeeper for brokers to discover each other and elect partition leaders. Kafka high-level consumers use ZK to divide up topic partitions amongst themselves. I'm not able to see how Consul +/or confd will help. Cheers, Roger On Fri, Oct 17, 2014 at 5:16 AM, Sybrandy, Casey < casey.sybra...@six3systems.com> wrote: > Neha, > > Thanks. I'd still love to know if anyone has used Consul and/or Confd to > manage a cluster. > > Casey > > ________________________________________ > From: Neha Narkhede [neha.narkh...@gmail.com] > Sent: Thursday, October 16, 2014 9:54 AM > To: users@kafka.apache.org > Subject: Re: Kafka/Zookeeper deployment Questions > > In other words, if I change the number of partitions, can I restart the > brokers one at a time so that I can continue processing data? > > Changing the # of partitions is an online operation and doesn't require > restarting the brokers. However, any other configuration (with the > exception of a few operations) that requires a broker restart can be done > in a rolling manner. > > On Wed, Oct 15, 2014 at 7:16 PM, Sybrandy, Casey < > casey.sybra...@six3systems.com> wrote: > > > Hello, > > > > We're looking into deploying Kafka and Zookeeper into an environment > where > > we want things to be as easy to stand up and administer. To do this, > we're > > looking into using Consul, or similar, and Confd to try to make this as > > automatic as possible. I was wondering if anyone had an experience in > this > > area. My major concern is reconfiguring Kafka as, in my experience, is > > making sure we don't end up losing messages. > > > > Also, can kafka and zookeeper be reconfigured in a rolling manner? In > > other words, if I change the number of partitions, can I restart the > > brokers one at a time so that I can continue processing data? > > > > Thanks. >
