Thanks, Ewen. I will try these.
-Zakee
On Sun, Feb 22, 2015 at 11:51 PM, Ewen Cheslack-Postava
wrote:
> If you haven't seen it yet, you probably want to look at
> http://kafka.apache.org/documentation.html#java
>
> -Ewen
>
> On Thu, Feb 19, 2015 at 10:53 AM, Zakee wrote:
>
> > Well are there a
Hi,
I'd start small and grow if needed in general. There is one bit of
info missing here. Did you try with a smaller heap and had issues? If so,
which issues? A few references from our internal Wiki:
- XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps
-XX:+PrintTenuringDistribution -XX:
If you haven't seen it yet, you probably want to look at
http://kafka.apache.org/documentation.html#java
-Ewen
On Thu, Feb 19, 2015 at 10:53 AM, Zakee wrote:
> Well are there any measurement techniques for Memory config in brokers. We
> do have a large load, with a max throughput 200MB/s. What
Well are there any measurement techniques for Memory config in brokers. We
do have a large load, with a max throughput 200MB/s. What do you suggest as
the recommended memory config for 5 brokers to handle such loads?
On Wed, Feb 18, 2015 at 7:13 PM, Jay Kreps wrote:
> 40G is really huge, general
40G is really huge, generally you would want more like 4G. Are you sure you
need that? Not sure what you mean by lsof and index files being too large,
but the index files are memory mapped so they should be able to grow
arbitrarily large and their memory usage is not counted in the java heap
(in fa
I am running a cluster of 5 brokers with 40G ms/mx for each. I found one of
the brokers is constantly using above ~90% of memory for jvm.heapUsage. I
checked from lsof output that the size of the index files for this broker
is too large.
Not sure what is going on with this one broker in the cluste
