RE: Abnormal memory consumption

2011-04-07 Thread Jeremiah Jordan
esday, April 06, 2011 7:59 PM To: user@cassandra.apache.org Subject: Re: Abnormal memory consumption Hello Paul, Thank you for the tip. The random port attribution policy of JMX was really making me mad ! Good to know there is a solution for that problem. Concerning the rest of the conversatio

Re: Abnormal memory consumption

2011-04-06 Thread openvictor Open
tuning guidelines is a guideline. You > can probably get acceptable performance with much less. If not, you can > shard your app such that you host a few Cfs per cluster. I doubt you'll need > to though. > > > From: openvictor Open > Reply-To: > Date: Mon, 4 Apr 2011 18:2

Re: Abnormal memory consumption

2011-04-06 Thread Paul Choi
rom: openvictor Open mailto:openvic...@gmail.com>> Reply-To: mailto:user@cassandra.apache.org>> Date: Mon, 4 Apr 2011 18:24:25 -0400 To: mailto:user@cassandra.apache.org>> Subject: Re: Abnormal memory consumption Okay, I see. But isn't there a big issue for scaling here ?

Re: Abnormal memory consumption

2011-04-05 Thread Peter Schuller
> Okay, I see. But isn't there a big issue for scaling here ? > Imagine that I am the developper of a certain very successful website : At > year 1 I need 20 CF. I might need to have 8Gb of RAM. Year 2 I need 50 CF > because I added functionalities to my wonderful webiste will I need 20 Gb of > RAM

Re: Abnormal memory consumption

2011-04-04 Thread openvictor Open
Okay, I see. But isn't there a big issue for scaling here ? Imagine that I am the developper of a certain very successful website : At year 1 I need 20 CF. I might need to have 8Gb of RAM. Year 2 I need 50 CF because I added functionalities to my wonderful webiste will I need 20 Gb of RAM ? And if

Re: Abnormal memory consumption

2011-04-04 Thread Peter Schuller
> And about the production 7Gb or RAM is sufficient ? Or 11 Gb is the minimum > ? > Thank you for your inputs for the JVM I'll try to tune that Production mem reqs are mostly dependent on memtable thresholds: http://www.datastax.com/docs/0.7/operations/tuning If you enable key caching or row

Re: Abnormal memory consumption

2011-04-04 Thread Victor Kabdebon
And about the production 7Gb or RAM is sufficient ? Or 11 Gb is the minimum ? Thank you for your inputs for the JVM I'll try to tune that 2011/4/4 Peter Schuller > > You can change VM settings and tweak things like memtable thresholds > > and in-memory compaction limits to get it down and get a

Re: Abnormal memory consumption

2011-04-04 Thread Peter Schuller
> You can change VM settings and tweak things like memtable thresholds > and in-memory compaction limits to get it down and get away with a > smaller heap size, but honestly I don't recommend doing so unless > you're willing to spend some time getting that right and probably > repeating some of the

Re: Abnormal memory consumption

2011-04-04 Thread Peter Schuller
> My last concern and for me it is a flaw for Cassandra and I am sad to admit > it because I love cassandra : how come that for 6Mb of data, Cassandra feels > the need to fill 500 Mb of RAM ? I can understand the need for, let's say, > 100 Mo because of cache and several Memtable being alive at the

Re: Abnormal memory consumption

2011-04-04 Thread openvictor Open
Hey Aaron, Thank you for your kind answer. This is a test server, the production serveur (single instance at the moment) has 8 Gb (or 12 Go not decided yet) of RAM. But with it there are other things running such as : Solr, Redis, PostGreSQL, Tomcat. The total take up to 1 Gb of RAM when running

Re: Abnormal memory consumption

2011-04-04 Thread aaron morton
For background see the JVM Heap Size section here http://wiki.apache.org/cassandra/MemtableThresholds You can also add a fudge factor of anywhere from X2 to X8 to the size of the memtables. You are in for a very difficult time trying to run cassandra with under 500MB of heap space. Is this ju

Abnormal memory consumption

2011-04-01 Thread openvictor Open
Hello everybody, I am quite new to Cassandra and I am worried about an apache cassandra server that is running on an small isolated server with only 2 Gb of RAM. On this server there is very little data in Cassandra ( ~3 Mb only text in column values) but there are other servers such as : SolR, T