Scalability question?

2014-06-26 Thread Zack Payton
Hi there, There have been some internal debates here about how far we can scale Kafka. Ideally, we'd be able to make it scale to 90 billion events a day. I've seen somewhere that linked scaled it up to 40 billion events a day. Has anyone seen a hard plateau in terms of scalability? Does anyone

Re: kafka.common.LeaderNotAvailableException

2014-06-25 Thread Zack
al Consultant > Big Data Open Source Security LLC > http://www.stealth.ly > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > ****/ > > >> On Tue, Jun 24, 2014 at 11:24 PM, Zack Payton wrote: >> >>

Re: kafka.common.LeaderNotAvailableException

2014-06-24 Thread Zack Payton
cipal Consultant > Big Data Open Source Security LLC > http://www.stealth.ly > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > ****/ > > > On Tue, Jun 24, 2014 at 9:56 PM, Zack Payton wrote: > >

kafka.common.LeaderNotAvailableException

2014-06-24 Thread Zack Payton
Hi all, I have 3 zookeeper servers and 2 Kafka servers. Running Kafka version 0.8.1.1. Running zookeeper 3.3.5-cdh3u6. >From the Kafka servers I can access the zookeeper servers on 2181. >From one of the Kafka servers I can create a topic no problem: [root@kafka1 kafka-0.8.1.1-src]# bin/kafka-topi