Yes I have, but I have to add that this is a server where there is so little data (2.0 Mo of text, rougly a book) that even if there were an overhead due to those things it would be minimal. I don't understand what's eating up all that memory, is it because of Linux that has difficulty getting rid of used memory ... I really am puzzled. (by the way it is not a Amazon EC2 server this is a dedicated server).
Regards, Victor K. 2011/2/8 Edward Capriolo <edlinuxg...@gmail.com> > On Tue, Feb 8, 2011 at 4:56 PM, Victor Kabdebon > <victor.kabde...@gmail.com> wrote: > > I will do that in the future and I will post my results here ( I upgraded > > the server to debian 6 to see if there is any change, so memory is back > to > > normal). I will report in a few days. > > In the meantime I am open to any suggestion... > > > > 2011/2/8 Aaron Morton <aa...@thelastpickle.com> > >> > >> When you attach to the JVM with JConsole how much non heap memory and > how > >> much heap memory is reported on the memory tab? > >> Xmx controls the total size of the heap memory, which excludes the > >> permanent generation. > >> see > >> > >> > http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html#generation_sizing > >> and > >> > >> > http://blogs.suncom/jonthecollector/entry/presenting_the_permanent_generation > >> Total non-heap memory on a 0.7 box I have is around 27M. You numbers > seem > >> large but it would be interesting to know what the JVM is reporting. > >> Aaron > >> On 09 Feb, 2011,at 05:57 AM, Victor Kabdebon <victor.kabde...@gmail.com > > > >> wrote: > >> > >> Information on the system : > >> > >> Debian 5 > >> Jvm : > >> victor@testhost:~/database/apache-cassandra-0.6.6$ java -version > >> java version "1.6.0_22" > >> Java(TM) SE Runtime Environment (build 1.6.0_22-b04) > >> Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03, mixed mode) > >> > >> RAM : 2Go > >> > >> > >> 2011/2/8 Victor Kabdebon <victor.kabde...@gmail.com> > >>> > >>> Sorry Jonathan : > >>> > >>> So most of these informations were taken using the command : > >>> > >>> sudo ps aux | grep cassandra > >>> > >>> For the nodetool information it is : > >>> > >>> /bin/nodetool --host localhost --port 8081 info > >>> > >>> > >>> Regars, > >>> > >>> Victor K. > >>> > >>> > >>> 2011/2/8 Jonathan Ellis <jbel...@gmail.com> > >>> > >>>> I missed the part where you explained where you're getting your > numbers > >>>> from. > >>>> > >>>> > >>>> On Tue, Feb 8, 2011 at 9:32 AM, Victor Kabdebon > >>>> <victor.kabde...@gmail.com> wrote: > >>>> > It is really weird that I am the only one to have this issue. > >>>> > I restarted Cassandra today and already the memory compution is over > >>>> > the > >>>> > limit : > >>>> > > >>>> > root 1739 4.0 24.5 664968 494996 pts/4 SLl 15:51 0:12 > >>>> > /usr/bin/java -ea -Xms128M -Xmx256M -XX:+UseParNewGC > >>>> > -XX:+UseConcMarkSweepGC > >>>> > -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 > >>>> > -XX:MaxTenuringThreshold=1 > >>>> > -XX:CMSInitiatingOccupancyFraction=75 > >>>> > -XX:+UseCMSInitiatingOccupancyOnly > >>>> > -XX:+HeapDumpOnOutOfMemoryError > >>>> > -Dcom.sun.management.jmxremote.port=8081 > >>>> > -Dcom.sun.management.jmxremotessl=false > >>>> > -Dcom.sun.management.jmxremote.authenticate=false > >>>> > -Dstorage-config=bin/../conf -cp > >>>> > > >>>> > > bin/../conf:bin/../build/classes:bin/../lib/antlr-3.1.3.jar:bin/../lib/apache-cassandra-06.6.jar:bin/../lib/avro-1.2.0-dev.jar:bin/../lib/cassandra-javautils.jar:bin/../lib/clhm-production.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-collections-3.2.1.jar:bin/../lib/commons-io-1.4.jar:bin/../lib/commons-lang-2.4.jar:bin/../lib/commons-pool-1.5.4.jar:bin/../lib/google-collections-1.0.jar:bin/../lib/hadoop-core-0.20.1.jar:bin/../lib/hector-0.6.0-14.jar:bin/../lib/high-scale-lib.jar:bin/./lib/ivy-2.1.0.jar:bin/../lib/jackson-core-asl-1.4.0.jar:bin/../lib/jackson-mapper-asl-1.4.0.jar:bin/../lib/jline-0.9.94.jar:bin/../lib/jna.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-r917130.jar:bin/../lib/log4j-1.2.14.jar:bin/../lib/perf4j-0.9.12.jar:bin/../lib/slf4j-api-1.5.8.jar:bin/../lib/slf4j-log4j12-1.5.8.jar:bin/../lib/uuid-3.1.jar > >>>> > org.apache.cassandra.thrift.CassandraDaemon > >>>> > > >>>> > It is really an annoying problem if we cannot really foresee memory > >>>> > consumption. > >>>> > > >>>> > Best regards, > >>>> > Victor K > >>>> > > >>>> > 2011/2/8 Victor Kabdebon <victor.kabde...@gmail.com> > >>>> >> > >>>> >> Dear all, > >>>> >> > >>>> >> Sorry to come back again to this point but I am really worried > about > >>>> >> Cassandra memory consumption. I have a single machine that runs one > >>>> >> Cassandra server. There is almost no data on it but I see a crazy > >>>> >> memory > >>>> >> consumption and it doesn't care at all about the instructions... > >>>> >> Note that I am not using mmap, but "Standard", I use also JNA > (inside > >>>> >> lib > >>>> >> folder), i am running on debian 5 64 bits, so a pretty normal > >>>> >> configuration. > >>>> >> I also use Cassandra 0.6.8. > >>>> >> > >>>> >> > >>>> >> Here are the informations I gathered on Cassandra : > >>>> >> > >>>> >> 105 16765 0.1 34.1 1089424 687476 ? Sl Feb02 14:58I > >>>> >> think you are > >>>> >> /usr/bin/java -ea -Xms128M -Xmx256M -XX:+UseParNewGC > >>>> >> -XX:+UseConcMarkSweepGC > >>>> >> -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 > >>>> >> -XX:MaxTenuringThreshold=1 > >>>> >> -XX:CMSInitiatingOccupancyFraction=75 > >>>> >> -XX:+UseCMSInitiatingOccupancyOnly > >>>> >> -XX:+HeapDumpOnOutOfMemoryError > >>>> >> -Dcom.sunmanagement.jmxremote.port=8081 > >>>> >> -Dcom.sun.management.jmxremote.ssl=false > >>>> >> -Dcom.sun.management.jmxremote.authenticate=false > >>>> >> -Dstorage-config=bin/../conf -Dcassandra-foreground=yes -cp > >>>> >> > >>>> >> > bin/../conf:bin/../build/classes:bin/../lib/antlr-3.1.3.jar:bin/../lib/apache-cassandra-0.6.6.jar:bin/../lib/avro-1.20-dev.jar:bin/../lib/cassandra-javautils.jar:bin/../lib/clhm-production.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-collections-3.2.1.jar:bin/../lib/commons-io-1.4.jar:bin/../lib/commons-lang-2.4.jar:bin/../lib/commons-pool-1.5.4.jar:bin/../lib/google-collections-1.0.jar:bin/../lib/hadoop-core-0.20.1.jar:bin/../lib/hector-0.6.0-14.jar:bin/../lib/high-scale-lib.jar:bin/../lib/ivy-2.1.0.jar:bin/../lib/jackson-core-asl-1.4.0.jar:bin/../lib/jackson-mapper-asl-1.4.0.jar:bin/./lib/jline-0.9.94.jar:bin/../lib/jna.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-r917130.jar:bin/../lib/log4j-1.2.14.jar:bin/../lib/perf4j-0.9.12.jar:bin/../lib/slf4j-api-1.5.8.jar:bin/../lib/slf4j-log4j12-1.5.8.jar:bin/../lib/uuid-3.1.jar > >>>> >> org.apache.cassandra.thrift.CassandraDaemon > >>>> >> > >>>> >> result of nodetool info : > >>>> >> > >>>> >> 116024732779488843382476400091948985708 > >>>> >> Load : 1,94 MB > >>>> >> Generation No : 1296673772 > >>>> >> Uptime (seconds) : 467550 > >>>> >> Heap Memory (MB) : 120,26 / 253,94 > >>>> >> > >>>> >> > >>>> >> I have about 21 column families, none of them have a lot of > >>>> >> information ( > >>>> >> as you see I have 2 Mb of text which is really small). Even if I > set > >>>> >> Xmx at > >>>> >> 256 there is 687M of memory used. Where does this memory come from > ? > >>>> >> Bad > >>>> >> garbage collection ? Something that I ignore ? > >>>> >> Thank you for your help I really need to get rid of that problem. > >>>> >> > >>>> >> Best regards, > >>>> >> Victor Kabdebon > >>>> > > >>>> > > >>>> > >>>> > >>>> > >>>> -- > >>>> Jonathan Ellis > >>>> Project Chair, Apache Cassandra > >>>> co-founder of DataStax, the source for professional Cassandra support > >>>> http://www.datastax.com > >>> > >> > > > > > > Each of your 21 column families will have its own memtable if you have > the default memtable settings your memory usage will grow quite large > over time. Have you tuned down your memtable size? >
