This means that code waiting on the iterator will continue to wait forever if rebalance fails? No exception will be thrown from the iterator?
I assume from your message that the only way to tell if a rebalance has failed and consumers have stopped consuming is by monitoring the lag and restarting the consumer from "outside" the code? --------------------------------------------------------- What Jun described is correct, currently the consumer asynchronously fails rebalance. But you can monitor the consumer to detect when a rebalance operation fails since that will manifest as lag on the consumer. Please see this< https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Myconsumerseemstohavestopped,why? >to learn about consumer lag monitoring. Thanks, Neha On Wed, Mar 26, 2014 at 9:06 PM, Jun Rao <jun...@gmail.com> wrote: > Currently, there is no callback on rebalance failure. The consumer > will retry failed rebalances. If all retries fail, we just log the error. > > Thanks, > > Jun > > > On Wed, Mar 26, 2014 at 5:01 PM, Tom Amon <ta46...@gmail.com> wrote: > > > The pattern for creating and operating consumers that we use is to > > create the consumer connector, create the streams and then consume > > each stream > by > > waiting on the iterator. > > > > If a rebalance occurs and fails, how is the error raised to the consumer? > > Will I get an exception while waiting on the iterator? Is it > > swallowed > and > > the consumer is dead? > > > > Thanks. > > >
