Updated the docs with Todd's recommendation - http://kafka.apache.org/081/documentation.html#java
Thanks, Neha On Tue, May 6, 2014 at 9:40 AM, Jay Kreps <jay.kr...@gmail.com> wrote: > Hey Todd, > > Doc patch? :-) > > svn co http://svn.apache.org/repos/asf/kafka/site/081/ > > Don't stress about html or formatting, I'm happy to do that part. I would > love to give people more authoritative advice. Right now everything is a > bit obsolete and wrong. > > -Jay > > > On Mon, May 5, 2014 at 10:36 PM, Todd Palino <tpal...@linkedin.com> wrote: > > > I apologize for taking a couple days to jump in on this. We¹re currently > > running JDK 1.7 u51, and we¹ve switched over to the G1 collector. If you > > do this (and I highly recommend it), make sure you¹re on u51. We tried > out > > u21 in testing, but we had a number of problems with the GC > implementation > > in that version. > > > > Our tuning looks like this: > > -Xms4g -Xmx4g -XX:PermSize=48m -XX:MaxPermSize=48m -XX:+UseG1GC > > -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 > > > > For reference, I¹m looking at the stats on one of our busiest clusters > (at > > peak): > > - 15 brokers > > - 15.5k partitions (replication factor 2) > > > > - 400k messages/sec in > > - 70 MB/sec inbound, 400 MB/sec+ outbound > > > > The tuning looks fairly aggressive, but all of our brokers in that > cluster > > have a 90% GC pause time of about 21ms, and they¹re doing less than 1 > > young GC per second. We haven¹t seen a single full GC on those brokers in > > the last month, and previous to that I think we only saw them when I was > > messing around with the cluster in a very painful way, not under anything > > approaching normal traffic. > > > > -Todd > > > > > > On 5/1/14, 9:21 PM, "Neha Narkhede" <neha.narkh...@gmail.com> wrote: > > > > >The GC settings at http://kafka.apache.org/documentation.html#java are > > >old. > > >We meant to update the documentation with the new GC settings using the > G7 > > >collector, but we haven't gotten around to doing that. Let me reach out > to > > >our engineer, Todd Palino, who worked on tuning GC for Kafka at LinkedIn > > >to > > >see if we can update our docs. > > > > > >Thanks, > > >Neha > > > > > > > > >On Thu, May 1, 2014 at 9:02 PM, Jun Rao <jun...@gmail.com> wrote: > > > > > >> http://kafka.apache.org/documentation.html#java > > >> > > >> Thanks, > > >> > > >> Jun > > >> > > >> > > >> On Thu, May 1, 2014 at 12:19 PM, Cassa L <lcas...@gmail.com> wrote: > > >> > > >> > Hi, > > >> > I want to know what usually, are the JAVA_HEAP settings > recommended > > >>for > > >> > kafka servers in production. > > >> > > > >> > Thanks > > >> > LCassa > > >> > > > >> > > > > >
