Re: Blocking Behavior of Metadata Refresh on New Producer

2014-02-13 Thread Jay Kreps
Yeah this is a good one to discuss. Current send will block in two conditions: 1. You are beginning a new data batch and have run out of buffer memory and block.on.buffer.full=true 2. Regardless of block.on.buffer.full the first request for each topic will block on fetching metadata that contains

Blocking Behavior of Metadata Refresh on New Producer

2014-02-13 Thread Guozhang Wang
I think we have discussed this issue before but I just want to bring it up again to get confirmed. Currently when the send() call is triggered, first of all the cluster information will be fetched for the specified topic, and if such information is not available yet the send call will block on ref