Providing a non-Zero value (Zero is default), for
public ConsumerRecords<K, V> poll(long timeout)
Works fine for me with no gaps,but said so is just a work around.
Consumer is definitely picking up messages with some delay.
-Sam
> On 22-Jan-2016, at 11:54 am, Jason Gustafson <ja...@confluent.io> wrote:
>
> Hi Krzysztof,
>
> This is definitely weird. I see the data in the broker's send queue, but
> there's a delay of 5 seconds before it's sent to the client. Can you create
> a JIRA?
>
> Thanks,
> Jason
>
>
>
> On Thu, Jan 21, 2016 at 11:30 AM, Samya Maiti <samya.maiti2...@gmail.com>
> wrote:
>
>> +1, facing same issue.
>> -Sam
>>> On 22-Jan-2016, at 12:16 am, Krzysztof Ciesielski <
>> krzysztof.ciesiel...@softwaremill.com> wrote:
>>>
>>> Hello, I'm running into an issue with the new consumer in Kafka 0.9.0.0.
>>> Here's a runnable gist illustrating the problem:
>>> https://gist.github.com/kciesielski/054bb4359a318aa17561 (requires
>> Kafka on
>>> localhost:9092)
>>>
>>> Scenario description:
>>> First, a producer writes 500000 elements into a topic
>>> Then, a consumer starts to read, polling in a loop.
>>> When "max.partition.fetch.bytes" is set to a relatively small value, each
>>> "consumer.poll()" returns a batch of messages.
>>> If this value is left as default, the output tends to look like this:
>>>
>>> Poll returned 13793 elements
>>> Poll returned 13793 elements
>>> Poll returned 13793 elements
>>> Poll returned 13793 elements
>>> Poll returned 0 elements
>>> Poll returned 0 elements
>>> Poll returned 0 elements
>>> Poll returned 0 elements
>>> Poll returned 13793 elements
>>> Poll returned 13793 elements
>>>
>>> As we can see, there are weird "gaps" when poll returns 0 elements for
>> some
>>> time. What is the reason for that? Maybe there are some good practices
>>> about setting "max.partition.fetch.bytes" which I don't follow?
>>>
>>> Bests,
>>> Chris
>>
>>
