Do you see many rebalances in your consumer log? Thanks,
Jun On Tue, Mar 12, 2013 at 6:01 PM, Dragos Manolescu < dragos.manole...@servicenow.com> wrote: > Hu Jun -- > > I appreciate the fast turnaround. I use Zookeeper 3.4.5. > > I am not sure about the consumer GC: the exception happens when I fire up > the client's VM which shouldn't be a busy time for the GC--I've been using > autooffset.reset=largest so I assume that the client doesn't do a whole > lot upon starting up. Perhaps I am wrong about that? > > Thanks, > > -Dragos > > > On 3/12/13 11:53 AM, "Jun Rao" <jun...@gmail.com> wrote: > > >What's the ZK version? You should use 3.3.4 or above, which fixed some > >bugs > >that could cause rebalance to fail. > > > > Also, a typical reason for many rebalancing is the consumer side GC. If > >so, you will see logs saying sth like "expired session" for ZK. > > > >Thanks, > > > >Jun > > > > > >On Tue, Mar 12, 2013 at 9:47 AM, Dragos Manolescu < > >dragos.manole...@servicenow.com> wrote: > > > >> Hi -- > >> > >> I have a Scala Kafka consumer, written in the image of the Java code > >>from > >> the Quickstart doc. I use the consumer for prototyping and debugging so > >>I > >> start and stop it quite frequently. I noticed that every now and then, > >>upon > >> starting the consumer, a ConsumerRebalanceFailedException is thrown (see > >> below). What's causing the problem this exception is signaling and what > >>are > >> the recommended practices for avoiding it? I am using Kafka 0.7.2 that I > >> built with Scala 2.9.2. > >> > >> Thanks, > >> > >> -Dragos > >> > >> Exception in thread "main" > >>kafka.common.ConsumerRebalanceFailedException: > >> Kconsumer-group_mseac02jh0rjdkq4-1363106041719-5b75c065 can't rebalance > >> after 4 retries > >> at > >> > >>kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedReba > >>lance(ZookeeperConsumerConnector.scala:473) > >> at > >> > >>kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsume > >>rConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:733) > >> at > >> > >>kafka.consumer.ZookeeperConsumerConnector.consume(ZookeeperConsumerConnec > >>tor.scala:207) > >> at > >> > >>kafka.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperC > >>onsumerConnector.scala:137) > >> at > >> > >>com.servicenow.syseng.kafka.KConsumer$delayedInit$body.apply(KConsumer.sc > >>ala:69) > >> at scala.Function0$class.apply$mcV$sp(Function0.scala:34) > >> at > >>scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12) > >> at scala.App$$anonfun$main$1.apply(App.scala:60) > >> at scala.App$$anonfun$main$1.apply(App.scala:60) > >> at > >> > >>scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scal > >>a:59) > >> at scala.collection.immutable.List.foreach(List.scala:76) > >> at > >> > >>scala.collection.generic.TraversableForwarder$class.foreach(TraversableFo > >>rwarder.scala:30) > >> at scala.App$class.main(App.scala:60) > >> > >> > >> > >
