[ 
https://issues.apache.org/jira/browse/KAFKA-761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13597377#comment-13597377
 ] 

Lorenzo Alberton commented on KAFKA-761:
----------------------------------------

Some more details: this happens with a moderately high number of topics (a few 
hundreds), when the consumers connect and disconnect quite often to fetch a few 
batches at a time.
                
> Fetch request randomly returns 0 bytes when there's still data in the log
> -------------------------------------------------------------------------
>
>                 Key: KAFKA-761
>                 URL: https://issues.apache.org/jira/browse/KAFKA-761
>             Project: Kafka
>          Issue Type: Bug
>          Components: consumer
>    Affects Versions: 0.7.2
>         Environment: centos 6
>            Reporter: Lorenzo Alberton
>            Assignee: Neha Narkhede
>
> I can't track the bug down in the code, but the behaviour has been hitting us 
> for quite some time (on Kafka 0.7.2): sometimes, a new Simple Consumer 
> object, on a new Fetch Request, returns an empty byte buffer (or a number of 
> bytes considerably smaller than the buffer size), even if there's more data 
> on disk. 
> This happens randomly, with no errors, at different offsets in the same 
> topic. Different consumer groups processing the same topic might get stuck at 
> different offsets.
> There seems to be no correlation between empty responses and the offset, or 
> the buffer size, or the log file size. At first we thought it only happened 
> when reaching the end of a log file, before jumping to the next file, but it 
> doesn't seem to be the case. It often happens in the middle of a file.
> Simply retrying the Fetch Request multiple times works quite reliably, after 
> 2 or 3 empty fetches the consumer eventually gets unstuck (we check the size 
> of the queue to see if there's more data before retrying).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Reply via email to