On 4/27/15 11:15 AM, "Jay Kreps" wrote:
>
>The new producer is pretty new still so I suspect there is a fair amount
>of
>low-hanging performance work for anyone who wanted to take a shot at it.
I do see something. Will discuss it on a different email thread.
Hi Jay,
Does o.a.k.clients.tools.ProducerPerformance provide multi-thread test? I
did not find it.
I tweaked the test a little bit to make it multi-threaded and what I found
is that in a single thread case, with each message of 10 bytes, single
caller thread has ~2M messages/second throughput. 2
Hey Jiangjie,
Yeah, not sure the bottleneck. It maybe the sender or lock contention on
the writer threads. You could use top or one of the java tools to check out
the per-thread cpu usage.
The benchmarking I had done previously showed an ability to max out the 1G
network cards we had with a singl
Jay,
Its not evident how to switch between sync and async modes using this
new 'org.apache.kafka.clients.tools.ProducerPerformance'
AFAIKT it measures in async mode by default.
-roshan
On 4/24/15 3:23 PM, "Jay Kreps" wrote:
>That should work. I recommend using the performance tool cited i
That should work. I recommend using the performance tool cited in the blog
linked from the "performance" page of the website. That tool is more
accurate and uses the new producer.
On Fri, Apr 24, 2015 at 2:29 PM, Roshan Naik wrote:
> Can we use the new 0.8.2 producer perf tool against a 0.8.1 br
Can we use the new 0.8.2 producer perf tool against a 0.8.1 broker ?
-roshan
On 4/24/15 1:19 PM, "Jay Kreps" wrote:
>Do make sure if you are at all performance sensitive you are using the new
>producer api we released in 0.8.2.
>
>-Jay
>
>On Fri, Apr 24, 2015 at 12:46 PM, Roshan Naik
>wrote:
>
Do make sure if you are at all performance sensitive you are using the new
producer api we released in 0.8.2.
-Jay
On Fri, Apr 24, 2015 at 12:46 PM, Roshan Naik
wrote:
> Yes, I too notice the same behavior (with producer/consumer perf tool on
> 8.1.2) Š adding more threads indeed improved the p
Yes, I too notice the same behavior (with producer/consumer perf tool on
8.1.2) Š adding more threads indeed improved the perf a lot (both with and
without --sync). in --sync mode
batch size made almost no diff, larger events improved the perf.
I was doing some 8.1.2 perf testing with a 1 node b
Hi,
I ran some tests on our cluster by sending message from multiple clients
(machines). Each machine had about 40-100 threads per producer.
I thought of trying out having multiple producers per clients with each
producer receiving messages from say 10-15 threads. I actually did see an
increase i
Hi Jay,
Yes, we are producing from single process/jvm.
>From docs "The producer will attempt to batch records together into fewer
requests whenever multiple records are being sent to the same partition."
If I understand correctly, batching happens at topic/partition level, not
at Node level. rig
If you are talking about within a single process, having one producer is
generally the fastest because batching dramatically reduces the number of
requests (esp using the new java producer).
-Jay
On Fri, Apr 24, 2015 at 4:54 AM, Manikumar Reddy
wrote:
> We have a 2 node cluster with 100 topics.
11 matches
Mail list logo
