Background http://en.wikipedia.org/wiki/Memory-mapped_file
> Is it going to load only relevant pages per SSTable on read or is it going to > load an entire SSTable on first access? It will load what is requested, and maybe some additional data taking into account the amount of memory available for caches. > Say suppose compaction kicks in. Will it then evict hot MMapped pages for > read and substitute it with a lot of pages involving full SSTables? Some file access in cassandra, such as compaction, hints to the OS that the reads should not be cached. Technically is uses posix_fadvise if you want to look it up. Cheers ----------------- Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 5/12/2012, at 11:04 PM, Ravikumar Govindarajan <ravikumar.govindara...@gmail.com> wrote: > Thanks Aaron, > > I am not quite clear on how MMap loads SSTables other than the fact that it > kicks in only during a first-time access > > Is it going to load only relevant pages per SSTable on read or is it going to > load an entire SSTable on first access? > > Say suppose compaction kicks in. Will it then evict hot MMapped pages for > read and substitute it with a lot of pages involving full SSTables? > > -- > Ravi > > On Wed, Dec 5, 2012 at 1:22 AM, aaron morton <aa...@thelastpickle.com> wrote: >> Will MMapping data files be detrimental for reads, in this case? > No. > >> In general, when should we opt for MMap data files and what are the factors >> that need special attention when enabling the same? > mmapping is the default, so I would say use it until you have a reason not > to. > > mmapping will map the entire file, but pages of data are read into memory on > demand and purged when space is needed. > > Cheers > > ----------------- > Aaron Morton > Freelance Cassandra Developer > New Zealand > > @aaronmorton > http://www.thelastpickle.com > > On 4/12/2012, at 11:59 PM, Ravikumar Govindarajan > <ravikumar.govindara...@gmail.com> wrote: > >> Our current SSTable sizes are far greater than RAM. {150 Gigs of data, 32GB >> RAM}. Currently we run with mlockall and mmap_index_only options and don't >> experience swapping at all. >> >> We use wide rows and size-tiered-compaction, so a given key will definitely >> be spread across multiple sstables. Will MMapping data files be detrimental >> for reads, in this case? >> >> In general, when should we opt for MMap data files and what are the factors >> that need special attention when enabling the same? >> >> -- >> Ravi > >
