Thank you Ewan, I'll give it a shot, but I am still puzzled as to the reasoning behind implementing hasNext() in a manner that it can block. Isn't that counter to the whole point of the method? You either have it or you not. Blocking in the off chance one may have it in some time in the future simply means it's not there at the moment, so why not return false and let user retry?
Sent from my iPhone > On Mar 3, 2016, at 03:51, Ewen Cheslack-Postava <e...@confluent.io> wrote: > > Take a look at the consumer.timeout.ms setting if you don't want the > iterator to block indefinitely. > > And a better long term solution is to switch to the new consumer, but that > obviously requires much more significant code changes. The new consumer API > is a single-threaded poll-based API where you can always specify timeouts > to the consumer's poll() method (though it currently has some limitations > to how it enforces that timeout). > > -Ewen > > On Wed, Mar 2, 2016 at 11:24 AM, Oleg Zhurakousky < > ozhurakou...@hortonworks.com> wrote: > >> Guys >> >> We have a consumer deadlock and here is the relevant dump: >> >> "Timer-Driven Process Thread-10" Id=76 TIMED_WAITING on >> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@39775787 >> at sun.misc.Unsafe.park(Native Method) >> at >> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) >> at >> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) >> at >> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) >> at >> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:65) >> at >> kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) >> at >> kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) >> at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) >> . . . . . >> >> What worries me is the fact that ‘hasNext’ is essentially a blocking >> operation. I can’t seem to find a single reason when it would be useful, >> hence I am calling it a bug, but hopefully someone can clarify. >> Kafka version is 0.8.* >> >> Cheers >> Oleg > > > -- > Thanks, > Ewen
