Hello,
I'm going through a series of tests with Kafka 0.8 to see what the failure
cases are. One simple scenario is this:
1. Launch Zookeeper
2. Launch my consumer (written in scala) but don't launch a Kafka broker (I am
not using replication in my configuration).
What happens in this particular scenario is that my consumer hits the
createMessageStreams which internally makes 4 failed attempts and then it
raises a ConsumerRebalanceFailedException:
kafka.common.ConsumerRebalanceFailedException:
zfilter_vm-florin-ubuntu-d-12-1373675347482-ff93b841 can't rebalance after 4
retries
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:397)
at
kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:681)
at
kafka.consumer.ZookeeperConsumerConnector.consume(ZookeeperConsumerConnector.scala:205)
at
kafka.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:133)
at
kafka.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:126)
I was expecting a more specific exception - something like NoBrokerAvailable
instead of this. I can add a catch clause for this exception and retry the
operation in a loop until the broker comes online but I dislike the approach
because I am making an assumption here - that ConsumerRebalanceFailedException
means "no broker available". Is there a better way to do this? Do you have a
pattern for consumers for when brokers go offline and come back up?
Thanks a lot!
Florin
