[
https://issues.apache.org/jira/browse/KAFKA-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14323107#comment-14323107
]
Jay Kreps commented on KAFKA-1956:
----------------------------------
Gotcha. Yes, that is just a mismatch of assumptions I think. There is no
guarantee that any particular poll will return anything. For example the server
that has the records could be undergoing leader election, or you could be
establishing a connection. Currently that method guarantees that it won't block
for much longer than the poll time you specify but doesn't guarantee it will
wait that full time.
However I think your complaint is that if you specify a higher timeout, say 5
seconds, you expect that it will wait the full 5 seconds for connections to be
established and records to be read. Instead it does the work of initiating
connections and then immediately returns control to you, expecting that you
will poll again. This breaks any attempt to use poll outside of a loop, a use
case I hadn't considered.
I agree we can fix this. However there are several problems with the patch, the
first issue is that it doesn't check the timeout so you may actually wait
indefinitely if no messages are being published and you are at the end of the
log. The other concern is that I'm not sure that it is safe to wait like that
in the fetch section, since if a partition reassignment occurs you will block
in that loop rather than picking up your new assignment. I think a safer fix
would be to repeat the whole poll loop until the timeout has expired.
I recommend we hold off on this change until the overall producer + server
implementation stabalizes a bit.
> new consumer client can't fetch records from kafka server
> ---------------------------------------------------------
>
> Key: KAFKA-1956
> URL: https://issues.apache.org/jira/browse/KAFKA-1956
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Affects Versions: 0.8.3
> Reporter: Hongliang Su
> Assignee: Neha Narkhede
> Labels: patch
> Attachments:
> 0005-fix-new-consumer-client-can-t-fetch-records-from-kaf.patch
>
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> new consumer client can't fetch records from kafka server
> suggestion:
> fetch response doesn't return even after one completeAll invoke
> block until the fetch response returns
> TODO: whether there is an effective way to wait until fetch response returns
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
