Kafka cluster with lots of topics

2013-11-13 Thread Joe Freeman
Would I be correct in assuming that a Kafka cluster won't scale well to support lots (tens of millions) of topics? If I understand correctly, a node being added or removed would involve a leader election for each topic, which is a relatively expensive operation?

Re: Kafka cluster with lots of topics

2013-11-14 Thread Joe Freeman
gt; > At those many topics, zookeeper will be the main bottleneck. Leader > > election process will take very long increasing the unavailability window > > of the cluster. > > > > Thanks, > > Neha > > On Nov 13, 2013 4:49 AM, "Joe Freeman" wrote: >