Unfortunately I've recompiled and installed kafka from scratch today, deleting all data. I'm still doing these tests, will send you stacktrace if I would be able to repro it.
Thanks for help! On Fri, Nov 1, 2013 at 6:13 PM, Neha Narkhede <neha.narkh...@gmail.com> wrote: > Yes, could you send around the log4j file of the consumer around the time > of the error in question. > > Thanks, > Neha > > > On Fri, Nov 1, 2013 at 11:35 AM, Kane Kane <kane.ist...@gmail.com> wrote: > >> Hello Neha, rest of the log goes to my consumer code, are you interested >> in? It's a little bit modified version of ConsoleConsumer. >> >> Thanks. >> >> >> On Fri, Nov 1, 2013 at 9:24 AM, Neha Narkhede <neha.narkh...@gmail.com >> >wrote: >> >> > Could you send the entire consumer log? >> > >> > >> > On Fri, Nov 1, 2013 at 7:45 AM, Kane Kane <kane.ist...@gmail.com> wrote: >> > >> > > Hello Neha, I think i might be hitting this. I didn't shutdown the >> > consumer >> > > (at least intentionally). Basically it was just an attempt to pipe ~1T >> > > through kafka, i would wild guess it's related to log expansion. >> Because >> > > around the time it happened i saw messages about expanding log, is it >> > > possible to be related? >> > > >> > > Thanks. >> > > >> > > >> > > On Fri, Nov 1, 2013 at 3:50 AM, Neha Narkhede <neha.narkh...@gmail.com >> > > >wrote: >> > > >> > > > I think you are hitting >> > https://issues.apache.org/jira/browse/KAFKA-824. >> > > > Was the consumer being shutdown at that time? >> > > > >> > > > >> > > > On Fri, Nov 1, 2013 at 1:28 AM, Kane Kane <kane.ist...@gmail.com> >> > wrote: >> > > > >> > > > > I've got the following exception running produce/consume loop for >> > > several >> > > > > hours, >> > > > > that was just single exception, but during that time both producers >> > and >> > > > > consumers slowed down a lot. After that looks like everything works >> > > fine, >> > > > > though i have >> > > > > suspicion some messages were lost. >> > > > > Can anyone explain what it might be? >> > > > > >> > > > > WARN consumer.ZookeeperConsumerConnector - >> > > > > [group2_kafka-001.sendgrid.net-1383289 >> > > > > 653031-b02bae92], exception during commitOffsets >> > > > > java.lang.NullPointerException >> > > > > at >> > > > > >> > > > >> > > >> > >> org.I0Itec.zkclient.ZkConnection.writeDataReturnStat(ZkConnection.java:115) >> > > > > at org.I0Itec.zkclient.ZkClient$10.call(ZkClient.java:817) >> > > > > at >> > > > > org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675) >> > > > > at >> > > > > org.I0Itec.zkclient.ZkClient.writeDataReturnStat(ZkClient.java:813) >> > > > > at >> org.I0Itec.zkclient.ZkClient.writeData(ZkClient.java:808) >> > > > > at >> org.I0Itec.zkclient.ZkClient.writeData(ZkClient.java:777) >> > > > > at kafka.utils.ZkUtils$.updatePersistentPath(Unknown >> Source) >> > > > > at >> > > > > >> > > > > >> > > > >> > > >> > >> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$3$$anonfun$apply$5.apply(Unknown >> > > > > Source) >> > > > > at >> > > > > >> > > > > >> > > > >> > > >> > >> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$3$$anonfun$apply$5.apply(Unknown >> > > > > Source) >> > > > > at >> > scala.collection.Iterator$class.foreach(Iterator.scala:727) >> > > > > at >> > > scala.collection.AbstractIterator.foreach(Iterator.scala:1157) >> > > > > at >> > > > > scala.collection.IterableLike$class.foreach(IterableLike.scala:72) >> > > > > at >> > scala.collection.AbstractIterable.foreach(Iterable.scala:54) >> > > > > at >> > > > > >> > > > > >> > > > >> > > >> > >> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$3.apply(Unknown >> > > > > Source) >> > > > > at >> > > > > >> > > > > >> > > > >> > > >> > >> kafka.consumer.ZookeeperConsumerConnector$$anonfun$commitOffsets$3.apply(Unknown >> > > > > Source) >> > > > > at >> > > > > >> > > > > >> > > > >> > > >> > >> scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772) >> > > > > at >> > scala.collection.Iterator$class.foreach(Iterator.scala:727) >> > > > > at kafka.utils.Pool$$anon$1.foreach(Unknown Source) >> > > > > at >> > > > > scala.collection.IterableLike$class.foreach(IterableLike.scala:72) >> > > > > at kafka.utils.Pool.foreach(Unknown Source) >> > > > > at >> > > > > >> > > > > >> > > > >> > > >> > >> scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771) >> > > > > at >> > > > kafka.consumer.ZookeeperConsumerConnector.commitOffsets(Unknown >> > > > > Source) >> > > > > >> > > > >> > > >> > >>
