Re: kafka.common.NotLeaderForPartitionException in Consumer

2015-06-22 Thread Jason Gustafson
I think this exception is logged by the consumer when there is a leadership change for a partition (which might be caused by one of the brokers going down or losing their session with zookeeper). Typically the consumer will recover after the cluster finds a new leader and the consumer updates its m

Re: kafka.common.NotLeaderForPartitionException in Consumer

2015-06-22 Thread Srividhya Anantharamakrishnan
ok, let me make it a little more clear. Datacenter A has 3 nodes, each acting as a broker, publishing messages to one of the nodes that has zookeeper running. Datacenter B has the same set up. Now, I am trying to publish message from one of the nodes in A to the ZK in A and make one of the nodes

Re: kafka.common.NotLeaderForPartitionException in Consumer

2015-06-22 Thread Jason Gustafson
Hi Srividhya, I'm a little confused about your setup. You have both clusters pointed to the same zookeeper, right? You don't appear to be using the zookeeper chroot option, so I think they would just form a single cluster. -Jason On Mon, Jun 22, 2015 at 3:50 PM, Srividhya Anantharamakrishnan < s

kafka.common.NotLeaderForPartitionException in Consumer

2015-06-22 Thread Srividhya Anantharamakrishnan
Hi, I have the following Kafka Set up - Two 3-node clusters A, B, where each node is acting as a broker and is connected by on ZK running in one node in cluster A. I was able to publish messages from cluster A and could subscribe to consuming from both A and B clusters. However, I am suddenly ru