Hello,
I did a test with these two rb scripts which take the time almost the
same. do you have the further idea?
$ cat async-pub.rb
require 'kafka'
kafka = Kafka.new("localhost:9092", client_id: "ruby-client",
resolve_seed_brokers: true)
producer = kafka.async_producer(required_acks: :all,max_buffer_size:
50_000,max_queue_size: 10_000)
10000.times do
message = rand.to_s
producer.produce(message, topic: "mytest")
end
producer.deliver_messages
producer.shutdown
$ cat sync-pub.rb
require 'kafka'
kafka = Kafka.new("localhost:9092", client_id: "ruby-client",
resolve_seed_brokers: true)
producer = kafka.producer(required_acks: :all,max_buffer_size: 50_000)
10000.times do
message = rand.to_s
producer.produce(message, topic: "mytest")
end
producer.deliver_messages
Thanks
On 2022/2/16 10:18, Luke Chen wrote:
Hi frakass,
I think the most difference for sync and async send (or "publish" like you
said), is the throughput.
You said the performance is almost the same, and I would guess the "acks"
config in your environment might be 0? Or maybe the produce rate is slow?
Or "max.in.flight.requests.per.connection" is 1?
Usually, when "acks=all", you have to wait for the records completely
replicated into all brokers before server response in "sync" mode, which is
why the throughput will be slow.
Compared with async mode, the producer send will return immediately after
appending the records, and wait for the response in callback function, no
matter it's acks=0 or acks=all.
Hope that helps.
Luke
On Wed, Feb 16, 2022 at 9:10 AM frakass <capitnfrak...@free.fr> wrote:
for a producer, is there a principle that when to use sync publishing,
and when to use async publishing?
for the simple format messages, i have tested both, their performance
are almost the same.
Thank you.
frakass
