Your producer test uses a thread per core. Your consumer test uses a single
thread. A single thread is likely insufficient to get maximum throughput.
On Aug 30, 2013 8:46 AM, "Rafael Bagmanov" wrote:
> Bejamin, do you mean thread on a client side? I'm not quite getting
> what I'm limited with. Ca
It is hard to say where the bottleneck is just from your description. Would
it be possible for you to rerun the consumer test using hprof on the
consumer so we can understand whether the fetcher is waiting on the fetches
(i.e. the broker is the bottleneck) or on the enque (i.e. the consumer is
the
Bejamin, do you mean thread on a client side? I'm not quite getting
what I'm limited with. Can you please explain little bit more?
A single threaded producer is still capable of doing 50 MB/s on hi1.4xlarge.
Which is quite slower than 377 MB/s from single job of FIO. But still
5 times faster than
I'm tried two different deployments:
1) Clients and broker on the same host (all the results I've shown are
for this configuration)
2) Client and broker on different hosts with 1 Gbits/s network channel
bandwidth between them (verified with iperf)
The results are practically the same.
Except that
You are maxing out the single consumer thread.
On Aug 30, 2013 1:35 AM, "Rafael Bagmanov" wrote:
> Hi,
>
> I am trying to understand how fast is kafka 0.7 compared to what I can get
> from hard drive. In essence I have 3 questions.
>
> In all tests below, I'm using single broker with single one-p
Are the clients on the same host as the broker? Could network be the
bottleneck?
Thanks,
Jun
On Fri, Aug 30, 2013 at 1:34 AM, Rafael Bagmanov wrote:
> Hi,
>
> I am trying to understand how fast is kafka 0.7 compared to what I can get
> from hard drive. In essence I have 3 questions.
>
> In all
