Hi, our cluster is deployed on AWS, we have brokers on r3.large instances, a decent amount of topics+partitions (+600 partitions). We're not making that many requests/sec, roughly 80 produce/sec and 240 fetch/sec (not counting internal replication requests) and yet CPU hovers around 40%, which I consider quite high given the nature of Kafka. I have worked on other deployments not on AWS where we were getting much larger figures in requests/sec, w/ much less CPU usage than that.
There are two things I consider trying to reduce that. The first one is obviously using 0.8.2.1 though I am not sure how much the impact is. I also found this thread from a while ago http://mail-archives.apache.org/mod_mbox/kafka-users/201305.mbox/%3ceb51b84c-ad91-4a2f-b97d-5283ef079...@transpac.com%3E about the use of the OpenJDK (which we're using, Ubuntu trusty's default) vs Oracle JDK. I am planning to do both anyways but I thought it'd be interesting to know if anyone else has experienced that before. Is there any other tuning I should think about? thanks marc
