Hi Tom, Currently we’re using the default settings – no special tuning whatsoever. I think the kafka-run-class.sh has this:
# Memory options if [ -z "$KAFKA_HEAP_OPTS" ]; then KAFKA_HEAP_OPTS="-Xmx256M" fi # JVM performance options if [ -z "$KAFKA_JVM_PERFORMANCE_OPTS" ]; then KAFKA_JVM_PERFORMANCE_OPTS="-server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC -Djava.awt.headless=true" fi Is this the confluent doc you were referring to? http://docs.confluent.io/2.0.0/kafka/deployment.html Thanks! Lawrence Weikum On 6/9/16, 1:32 PM, "Tom Crayford" <tcrayf...@heroku.com> wrote: >Hi Lawrence, > >What JVM options were you using? There's a few pages in the confluent docs >on JVM tuning iirc. We simply use the G1 and a 4GB Max heap and things work >well (running many thousands of clusters). > >Thanks >Tom Crayford >Heroku Kafka > >On Thursday, 9 June 2016, Lawrence Weikum <lwei...@pandora.com> wrote: > >> Hello all, >> >> We’ve been running a benchmark test on a Kafka cluster of ours running >> 0.9.0.1 – slamming it with messages to see when/if things might break. >> During our test, we caused two brokers to throw OutOfMemory errors (looks >> like from the Heap) even though each machine still has 43% of the total >> memory unused. >> >> I’m curious what JVM optimizations are recommended for Kafka brokers? Or >> if there aren’t any that are recommended, what are some optimizations >> others are using to keep the brokers running smoothly? >> >> Best, >> >> Lawrence Weikum >> >>
