The high-level consumer will block if a) there are no more messages available, b) the next message available is larger than the maximum fetch size you have specified, or c) your client code simply stops pulling messages from the iterator (the blocking queue will fill up).
Not sure if this applies to what you are seeing, but you should be aware of these conditions. Philip On Thu, Aug 22, 2013 at 12:10:54AM +0800, Frank Yao wrote: > Hi, > > My java consumer client will suddenly stop consuming the stream. While I > enabled the 'TRACE' log, I found that this stop always happened in the end > of a queuechunk consumption - after several times' chunk actually. What > weird is that in log there are not any useful information. It occurred > suddenly. > > I use jstack to figure out what happened and I found that perhaps the > thread is locked: > java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x00000000fa399970> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399) > at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:60) > at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:32) > at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:59) > at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:51 > > After google, I found some body has met this problem before, but no > solution there. > http://mail-archives.apache.org/mod_mbox/kafka-users/201211.mbox/%3CCANZjK9i87enoPY15rzh2Bg4D8+H1jvkSCkro=f3EROjn_4T=r...@mail.gmail.com%3E > > If any one can offer any help, I'll really appreciate it. > > ----------------- > > Frank Yao > @VIPSHOP, Shanghai, China -- Philip O'Toole Senior Developer Loggly, Inc. San Francisco, CA. www.loggly.com Come join us! http://loggly.com/company/careers/
