Re: Disk usage for CommitLog

2011-08-30 Thread Derek Andree
Okay I figured this out, the default for MemtableFlushAfterMins is not 60 minutes like some here said and what datastax docs say (http://www.datastax.com/docs/0.8/configuration/storage_configuration), it's 24 hours (1440). I changed them all to 60 for every CF and now commit logs only hang aro

Re: Disk usage for CommitLog

2011-08-30 Thread Derek Andree
> > 86GB in commitlog and 42GB in data > > Whoa, that seems really wrong, particularly given your data spans 13 months. > Have you changed any of the default cassandra.yaml setting? What is the > maximum memtable_flush_after across all your CFs? Any warnings/errors in the > Cassandra log? >

RE: Disk usage for CommitLog

2011-08-30 Thread Dan Hendry
August-29-11 23:20 To: user@cassandra.apache.org Subject: Re: Disk usage for CommitLog Thanks Dan, good info. > First off, what version of Cassandra are you using? Sorry my bad, 0.8.4 > Provided you are using a recent Cassandra version (late 0.7 or 0.8.x) I doubt the commit log is your

Re: Disk usage for CommitLog

2011-08-29 Thread Derek Andree
Thanks Dan, good info. > First off, what version of Cassandra are you using? Sorry my bad, 0.8.4 > Provided you are using a recent Cassandra version (late 0.7 or 0.8.x) I doubt > the commit log is your problem. My experience using Cassandra as a time > series data store (with a full 30 days of

Re: Disk usage for CommitLog

2011-08-29 Thread Jonathan Ellis
On Mon, Aug 29, 2011 at 8:04 PM, Derek Andree wrote: > Is there an easy way to keep commit log size down without killing performance? There is in 1.0: commitlog_total_space_in_mb, added in https://issues.apache.org/jira/browse/CASSANDRA-2427. Until then you'll have to make sure that you're flush

Re: Disk usage for CommitLog

2011-08-29 Thread Dan Hendry
First off, what version of Cassandra are you using? > We've noticed that when we restart cassandra disk utilization decreases dramatically Presumably you mean 'utilization' as in free space. Specifically on a restart, this type of behavior is likely due to Cassandra deleting compacted SSTables. C

Disk usage for CommitLog

2011-08-29 Thread Derek Andree
I run a single node cassandra instance, and we have lots of overwrites on a hot CF and disk utilization seems to grow pretty fast. We've noticed that when we restart cassandra disk utilization decreases dramatically (dramatic being something close to 50%). Most of this growth seems to be in th