It's worth noting that we currently run kakfa at LinkedIn with a 5G heap (not 3G, still using the CMS GC though - should update that), and the info on that wiki is aimed at 0.7.
We are actively working on things for 0.8 - don't have a 'this works for us', much less a 'recommendedation' there (hopefully will have something on that side soon). Every thing else on that wiki looks to be in the right ballpark. -- Dave DeMaagd ddema...@linkedin.com | 818 262 7958 (jay.kr...@gmail.com - Tue, Jan 22, 2013 at 07:35:17PM -0800) > We put a lot of info here: > https://cwiki.apache.org/confluence/display/KAFKA/Operations > > Does that help? > > -Jay > > > On Tue, Jan 22, 2013 at 7:14 PM, S Ahmed <sahmed1...@gmail.com> wrote: > > > In the wild, what sort of memory usage patterns have you guys seen with > > kafka? > > > > I'm not that well versed with java and its memory/gc issues, what have you > > guys experienced with your production brokers? (assuming consumers and > > producers are not that much of an issue). > > > > How do you determine how much memory to allocate to a given broker? Is it > > based on factors such as: > > > > 1. producers are mostly using batch versus single messages > > 2. messages per second > > 3. size of message payload > > 4. # of consumers connecting to a given broker > > etc. > > > > Is there a "magic" forumula? > >
