Re: Large number of files for Leveled Compaction

2013-06-18 Thread Franc Carter
dra to rebuild the sstables as bigger once I have updated the column family definition ? thanks > > cheers > > >> -Wei >> >> -- >> *From: *"Franc Carter" >> *To: *user@cassandra.apache.org >> *Sent: *Sunday, Jun

Re: Large number of files for Leveled Compaction

2013-06-17 Thread Eric Stevens
n 100MB. Do your own test o > find a "right" number. > > -Wei > > -- > *From: *"Franc Carter" > *To: *user@cassandra.apache.org > *Sent: *Sunday, June 16, 2013 10:15:22 PM > *Subject: *Re: Large number of files for Leveled Compaction &

Re: Large number of files for Leveled Compaction

2013-06-17 Thread Hiller, Dean
lt;mailto:user@cassandra.apache.org>" mailto:user@cassandra.apache.org>> Date: Sunday, June 16, 2013 11:37 PM To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" mailto:user@cassandra.apache.org>>, Wei Zhu mailto:wz1...@yahoo.com>> Subject: Re: L

Re: Large number of files for Leveled Compaction

2013-06-16 Thread Franc Carter
gt; find a "right" number. > Interesting - 50MB is the low end of what people are using - 5MB is a lot lower. I'll try a 50MB set cheers > -Wei > > -- > *From: *"Franc Carter" > *To: *user@cassandra.apache.org > *Sent: *Sun

Re: Large number of files for Leveled Compaction

2013-06-16 Thread Wei Zhu
Correction, the largest I heard is 256MB SSTable size. - Original Message - From: "Wei Zhu" To: user@cassandra.apache.org Sent: Sunday, June 16, 2013 10:28:25 PM Subject: Re: Large number of files for Leveled Compaction default value of 5MB is way too small in practice

Re: Large number of files for Leveled Compaction

2013-06-16 Thread Wei Zhu
age - From: "Franc Carter" To: user@cassandra.apache.org Sent: Sunday, June 16, 2013 10:15:22 PM Subject: Re: Large number of files for Leveled Compaction On Mon, Jun 17, 2013 at 2:59 PM, Manoj Mainali < mainalima...@gmail.com > wrote: Not in the case of LeveledCompac

Re: Large number of files for Leveled Compaction

2013-06-16 Thread Franc Carter
On Mon, Jun 17, 2013 at 2:59 PM, Manoj Mainali wrote: > Not in the case of LeveledCompaction. Only SizeTieredCompaction merges > smaller sstables into large ones. With the LeveledCompaction, the sstables > are always of fixed size but they are grouped into different levels. > > You can refer to th

Re: Large number of files for Leveled Compaction

2013-06-16 Thread Manoj Mainali
Not in the case of LeveledCompaction. Only SizeTieredCompaction merges smaller sstables into large ones. With the LeveledCompaction, the sstables are always of fixed size but they are grouped into different levels. You can refer to this page http://www.datastax.com/dev/blog/leveled-compaction-in-a

Re: Large number of files for Leveled Compaction

2013-06-16 Thread Franc Carter
On Mon, Jun 17, 2013 at 2:47 PM, Manoj Mainali wrote: > With LeveledCompaction, each sstable size is fixed and is defined by > sstable_size_in_mb in the compaction configuration of CF definition and > default value is 5MB. In you case, you may have not defined your own value, > that is why your ea

Re: Large number of files for Leveled Compaction

2013-06-16 Thread Franc Carter
On Fri, Jun 7, 2013 at 2:44 PM, Franc Carter wrote: > > Hi, > > We are trialling Cassandra-1.2(.4) with Leveled compaction as it looks > like it may be a win for us. > > The first step of testing was to push a fairly large slab of data into the > Column Family - we did this much faster (> x100) th

Re: Large number of files for Leveled Compaction

2013-06-16 Thread Manoj Mainali
With LeveledCompaction, each sstable size is fixed and is defined by sstable_size_in_mb in the compaction configuration of CF definition and default value is 5MB. In you case, you may have not defined your own value, that is why your each sstable is 5MB. And if you dataset is huge, you will see a l