Also, Could someone please explain how the factor of 7 comes in the picture in this sentence
"For example if memtable_total_space_in_mb is 100MB, and memtable_flush_writers is the default 1 (with one data directory), and memtable_flush_queue_size is the default 4, and a Column Family has no secondary indexes. The CF will not be allowed to get above one seventh of 100MB or 14MB, as if the CF filled the flush pipeline with 7 memtables of this size it would take 98MB. " On Wed, Jun 6, 2012 at 6:22 PM, rohit bhatia <rohit2...@gmail.com> wrote: > Hi.. > > the link http://thelastpickle.com/2011/05/04/How-are-Memtables-measured/ > mentions that "From version 0.7 onwards the worse case scenario is up > to CF Count + Secondary Index Count + memtable_flush_queue_size > (defaults to 4) + memtable_flush_writers (defaults to 1 per data > directory) memtables in memory the JVM at once.". > > So it implies that for flushing, Cassandra copies the memtables content. > So does this imply that writes to column families are not stopped even > when it is being flushed? > > Thanks > Rohit > > On Wed, Jun 6, 2012 at 9:42 AM, rohit bhatia <rohit2...@gmail.com> wrote: >> Hi Aaron >> >> Thanks for the link, I have gone through it. But this doesn't justify >> nodes of exactly same config/specs differing in their flushing >> frequency. >> The traffic on all node is same as we are using RandomPartitioner >> >> Thanks >> Rohit >> >> On Wed, Jun 6, 2012 at 12:24 AM, aaron morton <aa...@thelastpickle.com> >> wrote: >>> See the section on memtable_total_space_in_mb here >>> http://thelastpickle.com/2011/05/04/How-are-Memtables-measured/ >>> >>> Cheers >>> ----------------- >>> Aaron Morton >>> Freelance Developer >>> @aaronmorton >>> http://www.thelastpickle.com >>> >>> On 6/06/2012, at 2:27 AM, rohit bhatia wrote: >>> >>> I am trying to understand the variance in flushes frequency in a 8 >>> node Cassandra cluster. >>> All the flushes are of the same type and initiated by MeteredFlusher.java => >>> >>> "INFO [OptionalTasks:1] 2012-06-05 06:32:05,873 MeteredFlusher.java >>> (line 62) flushing high-traffic column family CFS(Keyspace='Stats', >>> ColumnFamily='Minutewise_Channel_Stats') (estimated 501695882 bytes)" >>> [taken from system.log] >>> >>> Number of flushes for 1 column family vary from 6 flushes per day to >>> 24 flushes per day among nodes of same configuration and same >>> hardware. >>> Could you please throw light on the what conditions does >>> MeteredFlusher use to trigger memtable flushes. >>> Also how accurate is the estimated size in the above logfile entry. >>> >>> Regards >>> Rohit Bhatia >>> Software Engineer, Media.net >>> >>>
