If I run 3 brokers in a cluster on localhost the cpu usage is virtually
zero. Not sure why on other environments the minimum usage of each broker
is at least 13% (with zero producers/consumers), that doesn't sound normal.
On Thu, Mar 23, 2017 at 4:48 PM, Paul van der Linden
wrote:
> Doesn't seem
Doesn't seem to be the clients indeed. Maybe it already uses 13% of cpu on
maintaining the cluster. With no connections at all, except zookeeper and
the other 2 brokers. This is the cpu usage:
CPU SAMPLES BEGIN (total = 86359) Thu Mar 23 16:47:26 2017
rank self accum count trace method
1 8
One thing that you might want to check is the number of consumers that
are connected/consuming against this Kafka setup. We have consistently
noticed that the CPU usage of the broker is very high even with very few
consumers (around 10 Java consumers). There's even a JIRA for it. From
what I re
Thanks. I managed to get a cpu dump from staging.
The output:
THREAD START (obj=5427, id = 24, name="RMI TCP Accept-0",
group="system")
THREAD START (obj=5427, id = 21, name="main", group="main")
THREAD START (obj=5427, id = 25, name="SensorExpiryThread",
group="main")
THRE
1. may be you can monitor thread wise cpu usage and correlate with thread
dump
to identify the bottleneck
2. Broker config property connections.max.idle.ms is used to close
idle connections.
default is 10min.
On Thu, Mar 23, 2017 at 3:55 PM, Paul van der Linden
wrote:
> Hi,
>
> I deploye
Hi,
I deployed Kafka about a week ago, but there are a few problems with how
Kafka behaves.
The first is the surprisingly high resource usage, one this the memory
(1.5-2 GB for each broker, 3 brokers) although this might be normal. The
other one is the cpu usage, which starts with 20% minimum on e
Hi,
I deployed Kafka about a week ago, but there are a few problems with how
Kafka behaves.
The first is the surprisingly high resource usage, one this the memory
(1.5-2 GB for each broker, 3 brokers) although this might be normal. The
other one is the cpu usage, which starts with 20% minimum on e
