Thanks! I also found old thread and Ewen did reply to that. So I will just
re-post so somebody else might find it helpful:
Ewen:
...
EndToEndLatency works with a single message at a time. It produces the
message then waits for the consumer to receive it. This approach guarantees
there is no delay
If you use the perf test without any bound on throughput it will
always try to send data faster than it can go out and build up a queue
of unsent data. So e.g. if your buffer is 1MB each send will be
blocked on waiting for the full 1MB of queued data to clear out and
get sent. This makes sense if y
Ewen,
One more question. I mentioned that *kafka-run-class.sh
org.apache.kafka.clients.tools.ProducerPerformance* latency is a lot higher
than *kafka-run-class.sh kafka.tools.TestEndToEndLatency.*
Example:
*ProducerPerformance:*
5000 records sent, 337463.891364 records/sec (32.18 MB/sec), *
It is single threaded in the sense that you can not request that multiple
threads be used to call producer.send(). However, the producer has its own
internal thread for doing network IO. When you have such a simple producer,
depending on the size of messages you can saturate a 1Gbps link with a
sin
Thanks Ewen. Do you know if kafka-run-class.sh
org.apache.kafka.clients.tools.ProducerPerformance
is single threaded? Or is there any way to specify number of threads?
On Fri, Jan 8, 2016 at 1:24 PM, Ewen Cheslack-Postava
wrote:
> Ah, sorry, I missed the version number in your title. I think thi
Ah, sorry, I missed the version number in your title. I think this tool saw
some rearrangement in 0.9.0 and I was looking at the latest version.
Unfortunately it doesn't look like the old kafka.tools.ProducerPerformance
that is used in kafka-producer-perf-test.sh in 0.8.2.1 supports passing in
addi
Ewen,
I tried that before like this:
./kafka-producer-perf-test.sh --broker-list test:9092 --topics test8-3
--messages 200 --new-producer --message-size 200
--show-detailed-stats max.request.size=1000
and it does not work. It comletly ignore this option.
And --producer-props is not
Andrew,
kafka-producer-perf-test.sh is just a wrapper around
orga.apache.kafka.clients.tools.ProducerPerformance and all command line
options should be forwarded. Can you just pass a --producer-props to set
max.request.size to a larger value?
-Ewen
On Fri, Jan 8, 2016 at 7:51 AM, Andrej Vladimir
Hi!
I am testing Kafka's performance with large messages and would like to
specify maximum request size when I run kafka-producer-perf-test.sh:
./kafka-producer-perf-test.sh --broker-list "test1:9092" --topics test8-3
--messages 100 --new-producer --msage-size 110 --show-detailed-stats
I
