Re: Commitlog Disk Full

2011-05-19 Thread Jonathan Ellis
That's basically the approach I want to take in https://issues.apache.org/jira/browse/CASSANDRA-2427. On Thu, May 19, 2011 at 12:00 PM, Mike Malone wrote: > Just noticed this thread and figured I'd chime in since we've had similar > issues with the commit log growing too large on our clusters. Tu

Re: Commitlog Disk Full

2011-05-19 Thread Mike Malone
Just noticed this thread and figured I'd chime in since we've had similar issues with the commit log growing too large on our clusters. Tuning down the flush timeout wasn't really an acceptable solution for us since we didn't want to be constantly flushing and generating extra SSTables for no reaso

Re: Commitlog Disk Full

2011-05-17 Thread mcasandra
m the same node. I looked at the code and it looks like you should see something in the logs for those files. -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Commitlog-Disk-Full-tp6356797p6375353.html Sent from the cassandra-u...@incubator.apache.or

Re: Commitlog Disk Full

2011-05-17 Thread Sanjeev Kulkarni
4797. On Tue, May 17, 2011 at 10:49 AM, mcasandra wrote: > Do you see anything in log files? > > -- > View this message in context: > http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Commitlog-Disk-Full-tp6356797p6374234.html > Sent from the cassandra-u...@incubator.apache.org mailing list archive at > Nabble.com. >

Re: Commitlog Disk Full

2011-05-17 Thread mcasandra
Do you see anything in log files? -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Commitlog-Disk-Full-tp6356797p6374234.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.

Re: Commitlog Disk Full

2011-05-16 Thread Sanjeev Kulkarni
- min_compaction_threshold: Avoid minor compactions of less than this >>> number of sstable files >>>- max_compaction_threshold: Compact no more than this number of >>> sstable >>> files at once >>>- column_metadata: Metadata which describes columns of column family. >>>Supported format is [{ k:v, k:v, ... }, { ... }, ...] >>>Valid attributes: column_name, validation_class (see comparator), >>> index_type (integer), index_name. >>> >>> >>> -- >>> View this message in context: >>> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Commitlog-Disk-Full-tp6356797p6370913.html >>> Sent from the cassandra-u...@incubator.apache.org mailing list archive >>> at Nabble.com. >>> >> >> >

Re: Commitlog Disk Full

2011-05-16 Thread Sanjeev Kulkarni
}, { ... }, ...] >> Valid attributes: column_name, validation_class (see comparator), >> index_type (integer), index_name. >> >> >> -- >> View this message in context: >> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Commitlog-Disk-Full-tp6356797p6370913.html >> Sent from the cassandra-u...@incubator.apache.org mailing list archive at >> Nabble.com. >> > >

Re: Commitlog Disk Full

2011-05-16 Thread Sanjeev Kulkarni
umn_metadata: Metadata which describes columns of column family. >Supported format is [{ k:v, k:v, ... }, { ... }, ...] >Valid attributes: column_name, validation_class (see comparator), > index_type (integer), index_name. > > > -- > View this message in context:

Re: Commitlog Disk Full

2011-05-16 Thread mcasandra
/Commitlog-Disk-Full-tp6356797p6370913.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.

Re: Commitlog Disk Full

2011-05-16 Thread Sanjeev Kulkarni
see if that helps. > > -- > View this message in context: > http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Commitlog-Disk-Full-tp6356797p6362301.html > Sent from the cassandra-u...@incubator.apache.org mailing list archive at > Nabble.com. >

Re: Commitlog Disk Full

2011-05-13 Thread mcasandra
.nabble.com/Commitlog-Disk-Full-tp6356797p6362301.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.

Re: Commitlog Disk Full

2011-05-13 Thread Sanjeev Kulkarni
our write happen in bursts. So often times, clients write data as fast as they can. Conceivably one can write 5G in one hour. The other setting that we have is that our replication factor is 3 and we write using QUORUM. Not sure if that will affect things. On Fri, May 13, 2011 at 12:04 AM, Peter S

Re: Commitlog Disk Full

2011-05-13 Thread mcasandra
Is there a way to look at the actual size of memtable? Would that help? -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Commitlog-Disk-Full-tp6356797p6360001.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at

Re: Commitlog Disk Full

2011-05-13 Thread Peter Schuller
> I haven't explictly set a value for the memtable_flush_after_mins parameter. > Looks like the default is 60minutes. > I will try to play around this value to see if that fixes things. Is the amount of data in the commit log consistent with what you might have been writing during 60 minutes? Incl

Re: Commitlog Disk Full

2011-05-12 Thread Sanjeev Kulkarni
Hi Peter, Thanks for the response. I haven't explictly set a value for the memtable_flush_after_mins parameter. Looks like the default is 60minutes. I will try to play around this value to see if that fixes things. Thanks again! On Thu, May 12, 2011 at 11:41 AM, Peter Schuller < peter.schul...@inf

Re: Commitlog Disk Full

2011-05-12 Thread Peter Schuller
> I understand that cassandra periodically cleans up the commitlog directories > by generating sstables in datadir. Is there any way to speed up this > movement from commitog to datadir? commitlog_rotation_threshold_in_mb could cause problems if it was set very very high, but with the default of 1

Commitlog Disk Full

2011-05-12 Thread Sanjeev Kulkarni
Hey guys, I have a ec2 debian cluster consisting of several nodes running 0.7.5 on ephimeral disks. These are fresh installs and not upgrades. The commitlog is set to the smaller of the disks which is around 10G in size and the datadir is set to the bigger disk. The config file is basically the sam