Dana, I am sorry, but I can’t accept that as an answer. 
Regardless, the API exposed to the end user must never “block indefinitely”. 
And saying you have to move a few mountains to work around what most would 
perceive to be a design issue is not the acceptable answer.
I’ll raise the JIRA

Cheers
Oleg

> On Apr 11, 2016, at 4:25 PM, Dana Powers <dana.pow...@gmail.com> wrote:
> 
> If you wanted to implement a timeout, you'd need to wire it up in
> commitOffsetsSync and plumb the timeout from Coordinator.close() and
> Consumer.close(). That's your answer. Code changes required.
> 
> -Dana
> 
> On Mon, Apr 11, 2016 at 1:17 PM, Oleg Zhurakousky
> <ozhurakou...@hortonworks.com> wrote:
>> Dana
>> Everything your are saying does not answer my question of how to interrupt a 
>> potential deadlock artificially forced upon users of KafkaConsumer API.
>> I may be OK with duplicate messages, I may be OK with data loss and I am OK 
>> with doing an extra work to do all kind of things. I am NOT OK with getting 
>> stuck ok close() call when I really want my system that uses KafkaConsumer 
>> to exit. So Consumer.close(timeout) is what I was really asking about.
>> So, is there a way now to interrupt such block?
>> 
>> Cheers
>> Oleg
>> 
>>> On Apr 11, 2016, at 4:08 PM, Dana Powers <dana.pow...@gmail.com> wrote:
>>> 
>>> Not a typo. This happens because the consumer closes the coordinator,
>>> and the coordinator attempts to commit any pending offsets
>>> synchronously in order to avoid duplicate message delivery. The
>>> Coordinator method commitOffsetsSync will retry indefinitely unless a
>>> non-recoverable error is encountered. If you wanted to implement a
>>> timeout, you'd need to wire it up in commitOffsetsSync and plumb the
>>> timeout from Coordinator.close() and Consumer.close(). It doesn't look
>>> terribly complicated, but you should check on the dev list for more
>>> opinions.
>>> 
>>> -Dana
>>> 
>>> On Mon, Apr 11, 2016 at 12:45 PM, Oleg Zhurakousky
>>> <ozhurakou...@hortonworks.com> wrote:
>>>> The subject line is from the javadoc of the new KafkaConsumer.
>>>> Is this for real? I mean I am hoping the use of ‘indefinitely' is a typo.
>>>> In any event if it is indeed true, how does one break out of indefinitely 
>>>> blocking consumer.close() invocation?
>>>> 
>>>> Cheers
>>>> Oleg
>>> 
>> 
> 

Reply via email to