Re: commitOffsets() in multithreaded consumer process

2013-08-11 Thread Jun Rao
Yes, I was thinking that you can have something like a 100ms timeout. High volume topics won't hit this timeout, but the iterator will not block. Low volume topics will return from the hasNext() call no more than 100ms. Thanks, Jun On Sat, Aug 10, 2013 at 12:51 AM, Jan Rudert wrote: > Thank y

Re: commitOffsets() in multithreaded consumer process

2013-08-10 Thread Jan Rudert
Thank you! So I guess, you suggest a really really small timeout so that the other consuming threads don't get regularly blocked for the timeout period? My consumer use case does not allow having "longer" breaks because there are some high traffic topics. Thanks Jan 2013/8/10 Jun Rao > The co

Re: commitOffsets() in multithreaded consumer process

2013-08-09 Thread Jun Rao
The consumer has a config property called consumer.timeout.ms. By setting the value to a positive integer, a timeout exception is thrown to the consumer if no message is available for consumption after the specified timeout value. Thanks, Jun On Fri, Aug 9, 2013 at 9:25 AM, Jan Rudert wrote: >

commitOffsets() in multithreaded consumer process

2013-08-09 Thread Jan Rudert
Hi, I have an consumer application where I have a message stream per topic and one thread per stream. I will do a commitOffsets() when a global shared message counter is reaching a limit. I think I need to make sure that no thread is consuming while I call commitOffsets() to ensure that no concu