RE: Two Kafka Question

2014-11-24 Thread Sybrandy, Casey
Gwen, Thanks. 1. I had a feeling about zookeeper being the potential bottleneck, but I wasn't sure. 2. Good to know. From: Gwen Shapira [gshap...@cloudera.com] Sent: Monday, November 24, 2014 2:47 PM To: users@kafka.apache.org Subject: Re: Two

Re: Two Kafka Question

2014-11-24 Thread Gwen Shapira
Hi Casey, 1. There's some limit based on size of zookeeper nodes, not sure exactly where it is though. We've seen 30 node clusters running in production. 2. For your scenario to work, the new broker will need to have the same broker id as the old one - or you'll need to manually re-assign partiti

Two Kafka Question

2014-11-24 Thread Sybrandy, Casey
Hello, First, is there a limit to how many Kafka brokers you can have? Second, if a Kafka broker node fails and I start a new broker on a new node, is it correct to assume that the cluster will copy data to that node to satisfy the replication factor specified for a given topic? In other words