2010/12/1 Ying Tang <ivytang0...@gmail.com> > I'm confused , plz ingore the mail above. > Here is my confusion , > posterior to 0.6.6/0.7 , minor compaction and major compaction both > can clean out rows 'tagged' tombstones , and generate a new , without > tombstones , sstable . >
This is right. > And the tombstones remains in memory ,waiting to be removed by jvm gc . > Am i right? > No! Compactions merge several old sstables into one, and mark old sstables as deleted which will be deleted while jvm gc. SSTable are files on harddisk, nothing to do with memory. You'd better have a look at Google's bigtable paper. > > On Wed, Dec 1, 2010 at 9:10 PM, Ying Tang <ivytang0...@gmail.com> wrote: > >> 1. So posterior to 0.6.6/0.7 , minor compaction and major compaction >> both can clean out rows 'tagged' tombstones , this kind of clean out >> doesn't mead remove it from the disk permanently. >> The real remove is done by the jvm GC ? >> 2. The intence of compaction is merging multi sstables into one , clean >> out the tombstone , let the un-tombstones rows be into a new ordered >> sstable ? >> >> >> >> On Wed, Dec 1, 2010 at 7:30 PM, Sylvain Lebresne <sylv...@yakaz.com>wrote: >> >>> On Wed, Dec 1, 2010 at 12:11 PM, Ying Tang <ivytang0...@gmail.com> >>> wrote: >>> > And i have another question , what's the difference between minor >>> > compaction and major compaction? >>> >>> A major compaction is a compaction that compact *all* the SSTables of a >>> given >>> column family (compaction compacts one CF at a time). >>> >>> Before https://issues.apache.org/jira/browse/CASSANDRA-1074 >>> (introduced in 0.6.6 and >>> recent 0.7 betas/rcs), major compactions where the only ones that removed >>> the >>> tombstones (see http://wiki.apache.org/cassandra/DistributedDeletes) >>> and this is the >>> reason major compaction exists. Now, with #1074, minor compactions >>> should remove most >>> if not all tombstones, so major compaction are not or much less useful >>> (it may depend on your >>> workload though as minor can't always delete the tombstones). >>> >>> -- >>> Sylvain >>> >>> > >>> > On 12/1/10, Chen Xinli <chen.d...@gmail.com> wrote: >>> >> 2010/12/1 Ying Tang <ivytang0...@gmail.com> >>> >> >>> >>> Every time cassandra creates a new sstable , it will call the >>> >>> CompactionManager.submitMinorIfNeeded ? And if the number of >>> memtables is >>> >>> beyond MinimumCompactionThreshold , the minor compaction will be >>> called. >>> >>> And there is also a method named CompactionManager.submitMajor , and >>> the >>> >>> call relationship is : >>> >>> >>> >>> NodeCmd -- > NodeProbe -->StorageService.forceTableCompaction --> >>> >>> Table.forceCompaction -->CompactionManager.performMajor --> >>> >>> CompactionManager.submitMajor >>> >>> >>> >>> ColumnFamilyStore.forceMajorCompaction --> >>> CompactionManager.performMajor >>> >>> --> CompactionManager.submitMajor >>> >>> >>> >>> >>> >>> HintedHandOffManager >>> >>> --> CompactionManager.submitMajor >>> >>> >>> >>> So i have 3 questions: >>> >>> 1. Once a new sstable has been created , >>> >>> CompactionManager.submitMinorIfNeeded will be called , >>> minorCompaction >>> >>> maybe called . >>> >>> But when will the majorCompaction be called ? Just the NodeCmd ? >>> >>> >>> >> >>> >> Yes, majorCompaction must be called manually from NodeCmd >>> >> >>> >> >>> >>> 2. Which jobs will minorCompaction and majorCompaction do ? >>> >>> Will minorCompaction delete the data that have been marked as >>> deleted >>> >>> ? >>> >>> And how about the major compaction ? >>> >>> >>> >> >>> >> Compaction only mark sstables as deleted. Deletion will be done when >>> there >>> >> are full gc, or node restarted. >>> >> >>> >> >>> >>> 3. When gc be called ? Every time compaction been called? >>> >>> >>> >> >>> >> GC has nothing to do with compaction, you may mistake the two >>> conceptions >>> >> >>> >> >>> >>> >>> >>> >>> >>> >>> >>> -- >>> >>> Best regards, >>> >>> >>> >>> Ivy Tang >>> >>> >>> >>> >>> >>> >>> >>> >>> >> >>> >> >>> >> -- >>> >> Best Regards, >>> >> Chen Xinli >>> >> >>> > >>> > >>> > -- >>> > Best regards, >>> > >>> > Ivy Tang >>> > >>> >> >> >> >> -- >> Best regards, >> >> Ivy Tang >> >> >> >> > > > -- > Best regards, > > Ivy Tang > > > > -- Best Regards, Chen Xinli
