I apologize for taking a couple days to jump in on this. We¹re currently
running JDK 1.7 u51, and we¹ve switched over to the G1 collector. If you
do this (and I highly recommend it), make sure you¹re on u51. We tried out
u21 in testing, but we had a number of problems with the GC implementation
in that version.
Our tuning looks like this:
-Xms4g -Xmx4g -XX:PermSize=48m -XX:MaxPermSize=48m -XX:+UseG1GC
-XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35
For reference, I¹m looking at the stats on one of our busiest clusters (at
peak):
- 15 brokers
- 15.5k partitions (replication factor 2)
- 400k messages/sec in
- 70 MB/sec inbound, 400 MB/sec+ outbound
The tuning looks fairly aggressive, but all of our brokers in that cluster
have a 90% GC pause time of about 21ms, and they¹re doing less than 1
young GC per second. We haven¹t seen a single full GC on those brokers in
the last month, and previous to that I think we only saw them when I was
messing around with the cluster in a very painful way, not under anything
approaching normal traffic.
-Todd
On 5/1/14, 9:21 PM, "Neha Narkhede" <[email protected]> wrote:
>The GC settings at http://kafka.apache.org/documentation.html#java are
>old.
>We meant to update the documentation with the new GC settings using the G7
>collector, but we haven't gotten around to doing that. Let me reach out to
>our engineer, Todd Palino, who worked on tuning GC for Kafka at LinkedIn
>to
>see if we can update our docs.
>
>Thanks,
>Neha
>
>
>On Thu, May 1, 2014 at 9:02 PM, Jun Rao <[email protected]> wrote:
>
>> http://kafka.apache.org/documentation.html#java
>>
>> Thanks,
>>
>> Jun
>>
>>
>> On Thu, May 1, 2014 at 12:19 PM, Cassa L <[email protected]> wrote:
>>
>> > Hi,
>> > I want to know what usually, are the JAVA_HEAP settings recommended
>>for
>> > kafka servers in production.
>> >
>> > Thanks
>> > LCassa
>> >
>>
