[
https://issues.apache.org/jira/browse/KAFKA-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15174110#comment-15174110
]
Jiangjie Qin edited comment on KAFKA-3236 at 3/1/16 6:01 PM:
-------------------------------------------------------------
Hi Thomas,
>From the description of the ticket, I think your confusion is from the
>vagueness of the behavior when {{block.on.buffer.full}} is set to false. Is it
>still a problem for you after we remove {{block.on.buffer.full}}?
I agree that having separate settings of blocking gives finer tuning
granularity, but we found it might not be necessary for user.
In your use case, if {{block.on.buffer.full = false}} and {{max.block.ms > 0}}
is not a pure non-blocking mode because producer.send() can still block up to
max.bock.ms, right?
That is the exact rationale of having a single {{max.block.ms}} instead of
decoupling time blocking on buffer full and metadata, because they provides the
same guarantee from user's perspective: at most block for {{max.blocking.ms}}.
was (Author: becket_qin):
Hi Thomas,
>From the description of the ticket, I think your confusion is from the
>vagueness of the behavior when {{block.on.buffer.full}} is set to false. Is it
>still a problem for you after we remove {{block.on.buffer.full}}?
I agree that having separate setting of blocking gives finer tuning
granularity, but we found it might not necessary for user.
In your use case, if {{block.on.buffer.full = false}} and {{max.block.ms > 0}}
is not a pure non-blocking mode because producer.send() can still block up to
max.bock.ms, right?
That is the exact rationale of having a single {{max.block.ms}} instead of
decoupling time blocking on buffer full and metadata, because they provides the
same guarantee from user's perspective: at most block for {{max.blocking.ms}}.
> Honor Producer Configuration "block.on.buffer.full"
> ---------------------------------------------------
>
> Key: KAFKA-3236
> URL: https://issues.apache.org/jira/browse/KAFKA-3236
> Project: Kafka
> Issue Type: Improvement
> Components: producer
> Affects Versions: 0.9.0.0
> Reporter: Thomas Graves
> Assignee: Thomas Graves
>
> In Kafka-0.9, "max.block.ms" is used to control how long the following
> methods will block.
> KafkaProducer.send() when
> * Buffer is full
> * Metadata is unavailable
> KafkaProducer.partitionsFor() when
> * Metadata is unavailable
> However when "block.on.buffer.full" is set to false, "max.block.ms" is in
> effect whenever a buffer is requested/allocated from the Producer BufferPool.
> Instead it should throw a BufferExhaustedException without waiting for
> "max.block.ms"
> This is particulary useful if a producer application does not wish to block
> at all on KafkaProducer.send() . We avoid waiting on KafkaProducer.send()
> when metadata is unavailable by invoking send() only if the producer instance
> has fetched the metadata for the topic in a different thread using the same
> producer instance. However "max.block.ms" is still required to specify a
> timeout for bootstrapping the metadata fetch.
> We should resolve this limitation by decoupling "max.block.ms" and
> "block.on.buffer.full".
> * "max.block.ms" will be used exclusively for fetching metadata when
> "block.on.buffer.full" = false (in pure non-blocking mode )
> * "max.block.ms" will be applicable to both fetching metadata as well as
> buffer allocation when "block.on.buffer.full = true
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
