Hello! I inherited Kafka cluster which runs on AWS (I3.4xl instances). Each node ingests around 8k messages per second. Instance stats: Network in is around: 13 MB/s Network out is around: 50 MB/s
Kafka stats show slightly different picture: counter-bytes-in: 2.7 MB/s counter-bytes-out: 2.9 MB/s CPU is constantly at 80% (16 cores) . I'm trying to figure out what can be causing such a high CPU and why kafka JMX stats are so different from AWS network stats. GC is not looking bad, the only fishy part is - there are *14000 connections *opened to each server. Can it be causing high CPU? What is the best way to troubleshoot it? Thank you, Alex