Hi, We have 2 small EC2 instances acting as "Receivers" for SPM<http://sematext.com/spm/>. Each of them runs Apache, Jetty, and a Kafka broker with the following params:
java -Xmx512M -server -Dlog4j.configuration=file:/usr/lib/kafka/bin/../config/log4j.properties -cp :/usr/lib/kafka/bin/../project/boot/scala-2.8.0/lib/scala-compiler.jar:/usr/lib/kafka/bin/../project/boot/scala-2.8.0/lib/scala-library.jar:/usr/lib/kafka/bin/../core/target/scala_2.8.0/kafka-0.7.2.jar:/usr/lib/kafka/bin/../core/lib/*.jar:/usr/lib/kafka/bin/../perf/target/scala_2.8.0/kafka*.jar:/usr/lib/kafka/bin/../core/lib_managed/scala_2.8.0/compile/jopt-simple-3.2.jar:/usr/lib/kafka/bin/../core/lib_managed/scala_2.8.0/compile/log4j-1.2.15.jar:/usr/lib/kafka/bin/../core/lib_managed/scala_2.8.0/compile/snappy-java-1.0.4.1.jar:/usr/lib/kafka/bin/../core/lib_managed/scala_2.8.0/compile/zkclient-0.1.jar:/usr/lib/kafka/bin/../core/lib_managed/scala_2.8.0/compile/zookeeper-3.3.4.jar kafka.Kafka /usr/lib/kafka/config/server.properties Some charts to give you an idea: Producer Request rate: https://apps.sematext.com/spm-reports/s/O5nPXZqEW0 Producer Request latencies: https://apps.sematext.com/spm-reports/s/8PNUQwcGeF Message sizes: https://apps.sematext.com/spm-reports/s/QVjhnskm6v And the machines are >90% idle. Here's one of them: https://apps.sematext.com/spm-reports/s/46zOdLaeen There are disk writes, of course. Here's a vmstat output for a chance: $ vmstat 2 procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 0 0 143696 46148 11408 610908 0 0 0 0 0 0 5 3 89 2 8 0 143696 46004 11404 610944 0 0 0 2912 3060 4831 4 3 92 1 0 0 143696 46772 11412 610172 0 0 54 3528 3122 4518 7 2 90 1 1 0 143696 46924 11416 609644 0 0 0 2880 1609 1859 2 1 96 1 6 0 143696 46164 11416 610268 0 0 0 2718 1828 2034 2 1 97 1 2 0 143696 46452 11432 610588 0 0 0 3720 2371 2393 3 1 95 1 Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Fri, May 2, 2014 at 12:21 AM, Neha Narkhede <neha.narkh...@gmail.com>wrote: > The GC settings at http://kafka.apache.org/documentation.html#java are > old. > We meant to update the documentation with the new GC settings using the G7 > collector, but we haven't gotten around to doing that. Let me reach out to > our engineer, Todd Palino, who worked on tuning GC for Kafka at LinkedIn to > see if we can update our docs. > > Thanks, > Neha > > > On Thu, May 1, 2014 at 9:02 PM, Jun Rao <jun...@gmail.com> wrote: > > > http://kafka.apache.org/documentation.html#java > > > > Thanks, > > > > Jun > > > > > > On Thu, May 1, 2014 at 12:19 PM, Cassa L <lcas...@gmail.com> wrote: > > > > > Hi, > > > I want to know what usually, are the JAVA_HEAP settings recommended > for > > > kafka servers in production. > > > > > > Thanks > > > LCassa > > > > > >
