[
https://issues.apache.org/jira/browse/KAFKA-233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jun Rao resolved KAFKA-233.
---------------------------
Resolution: Fixed
Fix Version/s: 0.8
Fixed in 0.8.
> The producer's load balancing logic can send requests to dead brokers, when
> using the async producer option
> -----------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-233
> URL: https://issues.apache.org/jira/browse/KAFKA-233
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.7
> Reporter: Neha Narkhede
> Labels: newbie
> Fix For: 0.8
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> The ZK producer, when used with the async producer option does the following
> 1. Create a pool of async producers, one each for a broker registered under
> /broker/ids
> 2. On each send request, apply the Partitioner, to decide the broker and
> partition to send the data
> 3. Use the Async producer's send API to enqueue that data into the async
> producer's queue
> 4. When the data is dequeued by the ProducerSendThread, use the underlying
> sync producer to send it to the broker
> The load balancing decision is taken in step 2, before entering the queue.
> This leaves a window of error, equal to the queue length, when a broker can
> go down. When this happens, potentially, a queue worth of data can fail to
> reach a broker, and will be dropped by the EventHandler.
> To correct this, the Producer, with the async option, needs to be refactored
> to allow only a single queue to hold all requests. And the application of the
> Partitioner should be moved to the end of the queue, in the EventHandler.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
