Hi Jonathan, hi Edward, Jonathan : but it looks like mmaping wants to consume the entire memory of my server. It goes up to 1.7 Gb for a ridiculously small amount of data. Am I doing something wrong or is there something I should change to prevent this never ending increase of memory consumption ? Edward : I am not sure, I will try to see that tomorrow but my disk access mode is standard, not mmap.
Anyway thank you very much, Victor K. PS : here is some hours after the result of ps aux | grep cassandra root 19093 0.1 30.0 1243940 *605060* ? Sl Jan11 10:15 /usr/bin/java -ea -Xms128M *-Xmx512M* -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Dcom.sun.management.jmxremote.port=8081 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dstorage-config=bin/../conf -Dcassandra-foreground=yes -cp bin/../conf:bin/../build/classes:bin/../lib/antlr-3.1.3.jar:bin/../lib/apache-cassandra-0.6.6.jar:bin/../lib/avro-1.2.0-dev.jar:bin/../lib/cassandra-javautils.jar:bin/../lib/clhm-production.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-collections-3.2.1.jar:bin/../lib/commons-io-1.4.jar:bin/../lib/commons-lang-2.4.jar:bin/../lib/commons-pool-1.5.4.jar:bin/../lib/google-collections-1.0.jar:bin/../lib/hadoop-core-0.20.1.jar:bin/../lib/hector-0.6.0-14.jar:bin/../lib/high-scale-lib.jar:bin/../lib/ivy-2.1.0.jar:bin/../lib/jackson-core-asl-1.4.0.jar:bin/../lib/jackson-mapper-asl-1.4.0.jar:bin/../lib/jline-0.9.94.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-r917130.jar:bin/../lib/log4j-1.2.14.jar:bin/../lib/perf4j-0.9.12.jar:bin/../lib/slf4j-api-1.5.8.jar:bin/../lib/slf4j-log4j12-1.5.8.jar:bin/../lib/uuid-3.1.jar org.apache.cassandra.thrift.CassandraDaemon 2011/1/15 Jonathan Ellis <jbel...@gmail.com> > mmapping only consumes memory that the OS can afford to feed it. > > On Fri, Jan 14, 2011 at 7:29 PM, Edward Capriolo <edlinuxg...@gmail.com> > wrote: > > On Fri, Jan 14, 2011 at 2:13 PM, Victor Kabdebon > > <victor.kabde...@gmail.com> wrote: > >> Dear rajat, > >> > >> Yes it is possible, I have the same constraints. However I must warn > you, > >> from what I see Cassandra memory consumption is not bounded in 0.6.X on > >> debian 64 Bit > >> > >> Here is an example of an instance launch in a node : > >> > >> root 19093 0.1 28.3 1210696 570052 ? Sl Jan11 9:08 > >> /usr/bin/java -ea -Xms128M -Xmx512M -XX:+UseParNewGC > -XX:+UseConcMarkSweepGC > >> -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 > -XX:MaxTenuringThreshold=1 > >> -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly > >> -XX:+HeapDumpOnOutOfMemoryError -Dcom.sun.management.jmxremote.port=8081 > >> -Dcom.sun.management.jmxremote.ssl=false > >> -Dcom.sun.management.jmxremote.authenticate=false > >> -Dstorage-config=bin/../conf -Dcassandra-foreground=yes -cp > >> > bin/../conf:bin/../build/classes:bin/../lib/antlr-3.1.3.jar:bin/../lib/apache-cassandra-0.6.6.jar:bin/../lib/avro-1.2.0-dev.jar:bin/../lib/cassandra-javautils.jar:bin/../lib/clhm-production.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-collections-3.2.1.jar:bin/../lib/commons-io-1.4.jar:bin/../lib/commons-lang-2.4.jar:bin/../lib/commons-pool-1.5.4.jar:bin/../lib/google-collections-1.0.jar:bin/../lib/hadoop-core-0.20.1.jar:bin/../lib/hector-0.6.0-14.jar:bin/../lib/high-scale-lib.jar:bin/../lib/ivy-2.1.0.jar:bin/../lib/jackson-core-asl-1.4.0.jar:bin/../lib/jackson-mapper-asl-1.4.0.jar:bin/../lib/jline-0.9.94.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-r917130.jar:bin/../lib/log4j-1.2.14.jar:bin/../lib/perf4j-0.9.12.jar:bin/../lib/slf4j-api-1.5.8.jar:bin/../lib/slf4j-log4j12-1.5.8.jar:bin/../lib/uuid-3.1.jar > >> org.apache.cassandra.thrift.CassandraDaemon > >> > >> Look at the second bold value, Xmx indicates the maximum memory that > >> cassandra can use; it is set to be 512, so it could easily fit into 1 > Gb. > >> Now look at the first one, 570Mb > 512 Mb. Moreover if I come back in > one > >> day the first value will be even higher. Probably around 610 Mb. > Actually it > >> increases to the point where I need to restart it otherwise other > program > >> are shut down by Linux for cassandra to further expand its memory > usage... > >> > >> By the way it's a call to other cassandra users, am I the only one to > >> encounter this problem ? > >> > >> Best regards, > >> > >> Victor K. > >> > >> 2011/1/14 Rajat Chopra <rcho...@makara.com> > >>> > >>> Hello. > >>> > >>> > >>> > >>> According to JVM heap size topic at > >>> http://wiki.apache.org/cassandra/MemtableThresholds , Cassandra would > need > >>> atleast 1G of memory to run. Is it possible to have a running Cassandra > >>> cluster with machines that have less than that memory… say 512M? > >>> > >>> I can live with slow transactions, no compactions etc, but do not want > an > >>> OutOfMemory error. The reason for a smaller bound for Cassandra is that > I > >>> want to leave room for other processes to run. > >>> > >>> > >>> > >>> Please help with specific parameters to tune. > >>> > >>> > >>> > >>> Thanks, > >>> > >>> Rajat > >>> > >>> > >> > > > > -Xmx512M is not an overall memory limit. MMAP'ed files also consume > > memory. Try turning disk access mode to standard not (MMAP or > > MMAP_INDEX_ONLY). > > > > > > -- > Jonathan Ellis > Project Chair, Apache Cassandra > co-founder of Riptano, the source for professional Cassandra support > http://riptano.com >
