Hey Todd, Doc patch? :-)
svn co http://svn.apache.org/repos/asf/kafka/site/081/ Don't stress about html or formatting, I'm happy to do that part. I would love to give people more authoritative advice. Right now everything is a bit obsolete and wrong. -Jay On Mon, May 5, 2014 at 10:36 PM, Todd Palino <tpal...@linkedin.com> wrote: > I apologize for taking a couple days to jump in on this. We¹re currently > running JDK 1.7 u51, and we¹ve switched over to the G1 collector. If you > do this (and I highly recommend it), make sure you¹re on u51. We tried out > u21 in testing, but we had a number of problems with the GC implementation > in that version. > > Our tuning looks like this: > -Xms4g -Xmx4g -XX:PermSize=48m -XX:MaxPermSize=48m -XX:+UseG1GC > -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 > > For reference, I¹m looking at the stats on one of our busiest clusters (at > peak): > - 15 brokers > - 15.5k partitions (replication factor 2) > > - 400k messages/sec in > - 70 MB/sec inbound, 400 MB/sec+ outbound > > The tuning looks fairly aggressive, but all of our brokers in that cluster > have a 90% GC pause time of about 21ms, and they¹re doing less than 1 > young GC per second. We haven¹t seen a single full GC on those brokers in > the last month, and previous to that I think we only saw them when I was > messing around with the cluster in a very painful way, not under anything > approaching normal traffic. > > -Todd > > > On 5/1/14, 9:21 PM, "Neha Narkhede" <neha.narkh...@gmail.com> wrote: > > >The GC settings at http://kafka.apache.org/documentation.html#java are > >old. > >We meant to update the documentation with the new GC settings using the G7 > >collector, but we haven't gotten around to doing that. Let me reach out to > >our engineer, Todd Palino, who worked on tuning GC for Kafka at LinkedIn > >to > >see if we can update our docs. > > > >Thanks, > >Neha > > > > > >On Thu, May 1, 2014 at 9:02 PM, Jun Rao <jun...@gmail.com> wrote: > > > >> http://kafka.apache.org/documentation.html#java > >> > >> Thanks, > >> > >> Jun > >> > >> > >> On Thu, May 1, 2014 at 12:19 PM, Cassa L <lcas...@gmail.com> wrote: > >> > >> > Hi, > >> > I want to know what usually, are the JAVA_HEAP settings recommended > >>for > >> > kafka servers in production. > >> > > >> > Thanks > >> > LCassa > >> > > >> > >
