Here is one example. 12GB data, no load besides OpsCenter and perhaps 1-2 requests per minute. INFO [ScheduledTasks:1] 2013-12-29 01:03:25,381 GCInspector.java (line 119) GC for ParNew: 426400 ms for 1 collections, 2253360864 used; max is 4114612224
2014/1/22 Yogi Nerella <ynerella...@gmail.com> > Hi, > > Can you share the GC logs for the systems you are running problems into? > > Yogi > > > On Wed, Jan 22, 2014 at 6:50 AM, Joel Samuelsson < > samuelsson.j...@gmail.com> wrote: > >> Hello, >> >> We've been having problems with long GC pauses and can't seem to get rid >> of them. >> >> Our latest test is on a clean machine with Ubuntu 12.04 LTS, Java >> 1.7.0_45 and JNA installed. >> It is a single node cluster with most settings being default, the only >> things changed are ip-addresses, cluster name and partitioner (to >> RandomPartitioner). >> We are running Cassandra 2.0.4. >> We are running on a virtual machine with Xen. >> We have 16GB of ram and default memory settings for C* (i.e. heap size of >> 4GB). CPU specified as 8 cores by our provider. >> >> Right now, we have no data on the machine and no requests to it at all. >> Still we get ParNew GCs like the following: >> INFO [ScheduledTasks:1] 2014-01-18 10:54:42,286 GCInspector.java (line >> 116) GC for ParNew: 464 ms for 1 collections, 102838776 used; max is >> 4106223616 >> >> While this may not be extremely long, on other machines with the same >> setup but some data (around 12GB) and around 10 read requests/s (i.e. >> basically no load) we have seen ParNew GC for 20 minutes or more. During >> this time, the machine goes down completely (I can't even ssh to it). The >> requests are mostly from OpsCenter and the rows requested are not extremely >> large (typically less than 1KB). >> >> We have tried a lot of different things to solve these issues since we've >> been having them for a long time including: >> - Upgrading Cassandra to new versions >> - Upgrading Java to new versions >> - Printing promotion failures in GC-log (no failures found!) >> - Different sizes of heap and heap space for different GC spaces (Eden >> etc.) >> - Different versions of Ubuntu >> - Running on Amazon EC2 instead of the provider we are using now (not >> with Datastax AMI) >> >> Something that may be a clue is that when running the DataStax Community >> AMI on Amazon we haven't seen the GC yet (it's been running for a week or >> so). Just to be clear, another test on Amazon EC2 mentioned above (without >> the Datastax AMI) shows the GC freezes. >> >> If any other information is needed, just let me know. >> >> Best regards, >> Joel Samuelsson >> > >
