Cool.
By the way, I do mean you should use 'atop'. That was not a typo on my part.
http://www.atoptool.nl/downloadatop.php
apt-get install atop
on Ubuntu systems.
Philip
On May 21, 2013, at 4:51 PM, Jason Weiss wrote:
> Philip,
>
> Thanks for the response. I used top yesterday and deter
Philip,
Thanks for the response. I used top yesterday and determined that part of
my problem was that the kafaka shell script is pre-configured to only use
512M of RAM, and thus it wasn't using memory efficiently. That has helped
out tremendously. Adding an echo at the start of the script that it
As a test, why not just use a disk with provisioned IOPs of 4000? Just as a
test - see if it improves.
Also, you have not supplied any metrics regarding the VM's performance. Is the
CPU busy? Is IO maxed out? Network? Disk? Use a tool like atop, and tell us
what you find.
Philip
On May 20, 2
Hi Jason,
On May 20, 2013, at 10:01am, Jason Weiss wrote:
> Hi Scott.
>
> I'm using Kafka 0.7.2. I am using the default replication factor, since I
> don't recall changing that configuration at all.
>
> I'm using provisioned IOPS, which from attending the AWS event in NYC a
> few weeks ago was
Ahh, yeah, piops is definitely faster than standard EBS, but still much
slower than local disk.
you could try benchmarking local disk to see what the instances you are
using are capable of, then try tweaking iops etc to see where you get.
M1.Larges arent super fast so your macbook beating them
Hi Scott.
I'm using Kafka 0.7.2. I am using the default replication factor, since I
don't recall changing that configuration at all.
I'm using provisioned IOPS, which from attending the AWS event in NYC a
few weeks ago was presented as the "fastest storage option" for EC2. A
number of partners pr
My guess, EBS is likely your bottleneck. Try running on instance local
disks, and compare your results. Is this 0.8? What replication factor are
you using?
On Mon, May 20, 2013 at 8:11 AM, Jason Weiss wrote:
> I'm trying to maximize my throughput and seem to have hit a ceiling.
> Everything d
