On Tue, Oct 28, 2014 at 9:02 AM, Adria Arcarons < adria.arcar...@greenpowermonitor.com> wrote:
> Hi, > > Hi > > > We have about 50.000 CFs of varying size > > > > > The writing test consists of a continuous flow of inserts. The inserts are > done inside BATCH statements in groups of 1.000 to a single CF at a time to > make them faster. > > > > The problem I’m experiencing is that, eventually, when the script has been > running for almost 40mins, the heap gets saturated. OldGen gets full and > then there is an intensive GC activity trying to free OldGen objects, but > it can only free very little space in each pass. Then GC saturates the CPU. > Here are the graphs obtained with VisualVM that show this behavior: > > > > > > My total heap size is 1GB and the the NewGen region of 256MB. The C* node > has 4GB RAM. Intel Xeon CPU E5520 @ > Without looking at your VM graphs, I'm going to go out on a limb here and say that your host is woefully underpowered to host fifty-thousand column families and batch writes of one-thousand statements. A 1 GB java heap size is sometimes acceptable for a unit test or playing around with but you can't actually expect it to be adequate for a load test can you? Every CF consumes some permanent heap space for its metadata. Too many CF are a bad thing. You probably have ten times more CF than would be recommended as an upper limit. -Bryan
