GitHub user knusbaum opened a pull request:
https://github.com/apache/kafka/pull/929
KAFKA-3236: Honor Producer Configuration "block.on.buffer.full"
In Kafka-0.9, `max.block.ms` is used to control how long the following
methods will block.
`KafkaProducer.send()` when
- Buffer is full
- Metadata is unavailable
`KafkaProducer.partitionsFor()` when
- Metadata is unavailable
However when `block.on.buffer.full` is set to false, `max.block.ms` is in
effect whenever a buffer is requested/allocated from the Producer BufferPool.
Instead it should throw a BufferExhaustedException without waiting for
`max.block.ms`
This is particulary useful if a producer application does not wish to block
at all on `KafkaProducer.send()`. We avoid waiting on `KafkaProducer.send()`
when metadata is unavailable by invoking `send()` only if the producer instance
has fetched the metadata for the topic in a different thread using the same
producer instance. However `max.block.ms` is still required to specify a
timeout for bootstrapping the metadata fetch.
We should resolve this limitation by decoupling `max.block.ms` and
`block.on.buffer.full`.
- `max.block.ms` will be used exclusively for fetching metadata when
`block.on.buffer.full = false` (in pure non-blocking mode )
- `max.block.ms` will be applicable to both fetching metadata as well as
buffer allocation when `block.on.buffer.full = true`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/knusbaum/kafka KAFKA-3236
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/929.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #929
----
commit 8cc454d2bc9d2be1b0ee916a5dd76042b79954fb
Author: Sanjiv Raj <sanj...@yahoo-inc.com>
Date: 2016-02-11T22:20:26Z
[ADDHR-1240] Honor block.on.buffer.full producer configuration
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
