If you haven't seen it yet, you probably want to look at http://kafka.apache.org/documentation.html#java
-Ewen On Thu, Feb 19, 2015 at 10:53 AM, Zakee <kzak...@netzero.net> wrote: > Well are there any measurement techniques for Memory config in brokers. We > do have a large load, with a max throughput 200MB/s. What do you suggest as > the recommended memory config for 5 brokers to handle such loads? > > On Wed, Feb 18, 2015 at 7:13 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > > > 40G is really huge, generally you would want more like 4G. Are you sure > you > > need that? Not sure what you mean by lsof and index files being too > large, > > but the index files are memory mapped so they should be able to grow > > arbitrarily large and their memory usage is not counted in the java heap > > (in fact by having such a large heap you are taking away OS memory from > > them). > > > > -Jay > > > > On Wed, Feb 18, 2015 at 4:13 PM, Zakee <kzak...@netzero.net> wrote: > > > > > I am running a cluster of 5 brokers with 40G ms/mx for each. I found > one > > of > > > the brokers is constantly using above ~90% of memory for > jvm.heapUsage. I > > > checked from lsof output that the size of the index files for this > broker > > > is too large. > > > > > > Not sure what is going on with this one broker in the cluster? Why > would > > > the index file sizes be so hugely different on one broker? Any ideas? > > > > > > > > > Regards > > > Zakee > > > ____________________________________________________________ > > > Invest with the Trend > > > Exclusive Breakout Alert On Soaring Social Media Technology > > > > http://thirdpartyoffers.netzero.net/TGL3231/54e52a9fe121d2a9f4a27st01vuc > > ____________________________________________________________ > > Have you been injured? > > Get a free evaluation today to see what your injury case is worth. > > http://thirdpartyoffers.netzero.net/TGL3255/54e55ad9894265ad90bcbmp13duc > -- Thanks, Ewen
