Hi Aaron. I wanted to try the new config. After doing a rolling restart I have all my counters false, with wrong values. I stopped my servers with the following :
nodetool -h localhost disablegossip nodetool -h localhost disablethrift nodetool -h localhost drain kill cassandra sigterm (15) via htop And after restarting the second one I have lost all the consistency of my data. All my statistics since September are totally false now in production. As reminder I'm using a 2 node cluster RF=2, CL.ONE 1 - How to fix it ? (I have a backup from this morning, but I will lose all the data after this date if I restore this backup) 2 - What happened ? How to avoid it ? Any Idea would be greatly appreciated, I'm quite desperated. Alain 2012/5/17 aaron morton <aa...@thelastpickle.com>: > What is the the benefit of having more memory ? I mean, I don't > > understand why having 1, 2, 4, 8 or 16 GB of memory is so different. > > Less frequent and less aggressive garbage collection frees up CPU resources > to run the database. > > Less memory results in frequent and aggressive (i.e. stop the world) GC, and > increase IO pressure. Which reduces read performance and in the extreme can > block writes. > > The memory used inside > > the heap will remains close to the max memory available, therefore > having more or less memory doesn't matter. > > Not an ideal situation. Becomes difficult to find an contiguous region of > memory to allocate. > > Can you enlighten me about this point ? > > It's a database server, it is going to work better with more memory. Also > it's Java and it's designed to run on multiple machines with many GB's of > ram available. There are better arguments > here http://wiki.apache.org/cassandra/CassandraHardware > > > I'm interested a lot in learning about some configuration I can use to > reach better peformance/stability as well as in learning about how > Cassandra works. > > Turn off all caches. > > In the schema increase the bloom filter false positive rate (see help in the > cli for Create column family) > > In the yaml experiment with these changes: > * reduce sliced_buffer_size_in_kb > * reduce column_index_size_in_kb > * reduce in_memory_compaction_limit_in_mb > * increase index_interval > * set concurrent_compactors to 2 > > Cheers > > ----------------- > Aaron Morton > Freelance Developer > @aaronmorton > http://www.thelastpickle.com > > On 17/05/2012, at 12:40 AM, Alain RODRIGUEZ wrote: > > Using c1.medium, we are currently able to deliver the service. > > What is the the benefit of having more memory ? I mean, I don't > understand why having 1, 2, 4, 8 or 16 GB of memory is so different. > In my mind, Cassandra will fill the heap and from then, start to flush > and compact to avoid OOMing and fill it again. The memory used inside > the heap will remains close to the max memory available, therefore > having more or less memory doesn't matter. > > I'm pretty sure I misunderstand or forget something about how the > memory is used but not sure about what. > > Can you enlighten me about this point ? > > If I understand why the memory size is that important I will probably > be able to argue about the importance of having more memory and my > boss will probably allow me to spend more money to get better servers. > > "There are some changes you can make to mitigate things (let me know > if you need help), but this is essentially a memory problem." > > I'm interested a lot in learning about some configuration I can use to > reach better peformance/stability as well as in learning about how > Cassandra works. > > Thanks for the help you give to people and for sharing your knowledge > with us. I appreciate a lot the Cassandra community and the most > active people keeping it alive. It's worth being said :). > > Alain > >
