Hi Nimi I am not familiar with how CASSANDRA-5863 uses the buffer pool yet, but I do know how the buffer pool works
It has a local pool, which is thread local, and takes chunks from a global pool. These are allocated off heap and aligned to the page cache boundaries, see here <https://github.com/apache/cassandra/blob/30bb255ec9fb36ace2aab51474bd3bfb9bbd3bed/src/java/org/apache/cassandra/utils/memory/BufferPool.java#L276>. The buffer pool will allocate according to buffer_pool_use_heap_if_exhausted only in two cases: - The global pool has run out of chunks, which happens when there is an OOM exception or when MEMORY_USAGE_THRESHOLD is exhausted, see here <https://github.com/apache/cassandra/blob/30bb255ec9fb36ace2aab51474bd3bfb9bbd3bed/src/java/org/apache/cassandra/utils/memory/BufferPool.java#L259> . - The requested buffer size is larger than a chunk, currently CHUNK_SIZE, 64 kb. To understand this behaviour look at maybeTakeFromPool <https://github.com/apache/cassandra/blob/30bb255ec9fb36ace2aab51474bd3bfb9bbd3bed/src/java/org/apache/cassandra/utils/memory/BufferPool.java#L124>(), the line that takes from the local pool is the last one and follow from there. The original ticket for the buffer pool is CASSANDRA-8897 <https://issues.apache.org/jira/browse/CASSANDRA-8897>if you need more details, and there is CASSANDRA-9468 <https://issues.apache.org/jira/browse/CASSANDRA-9468> to extend it to allow chunks with bigger or smaller sizes (point 4). I've left a comment on CASSANDRA-11993. I hope this helps, Stefania On Wed, Jun 22, 2016 at 4:54 AM, Nimi Wariboko Jr <n...@channelmeter.com> wrote: > Hi, > > I'm trying to debug an issue I'm having with Cassandra 3.6+ and Snappy > compressed tables - CASSANDRA-11993 (that may be related to > CASSANDRA-5863). > > I'm probably 3% familiar with Cassandra internals, so apologies if I'm > wrong here - I was looking at a change in CASSANDRA-5863 where a cache was > added to the SSTableIterator code path, which may have > included org.apache.cassandra.utils.memory.BufferPool into the read path. > > My confusion comes from here - AFAICT, when the BufferPool needs to > allocate more memory, it calls allocate(int size, boolean onHeap) ( > > https://github.com/apache/cassandra/blob/30bb255ec9fb36ace2aab51474bd3bfb9bbd3bed/src/java/org/apache/cassandra/utils/memory/BufferPool.java#L82 > ). > The value for whether onHeap is true is > constant ALLOCATE_ON_HEAP_WHEN_EXAHUSTED. > > ALLOCATE_ON_HEAP_WHEN_EXAHUSTED is defined by the yaml > setting buffer_pool_use_heap_if_exhausted which is true by default (and was > so in my install). > > Further still ( > > https://github.com/apache/cassandra/blob/30bb255ec9fb36ace2aab51474bd3bfb9bbd3bed/src/java/org/apache/cassandra/utils/memory/BufferPool.java > ), > I can't see any logic where BufferPool will actually allocate off heap > memory - it seems, due to the buffer_pool_use_heap_if_exhausted setting, > always allocate memory on heap. It never checks if the off heap memory is > exhausted (AFAICT from these 3 files). > > Sure enough, if I set buffer_pool_use_heap_if_exhausted, my issue at > CASSANDRA-11993 goes away (I think? I'm having a number of issues lately > and I may have traded one exception for another). > > Is this the correct behavior of buffer_pool_use_heap_if_exhausted? > > Nimi > -- [image: datastax_logo.png] <http://www.datastax.com/> Stefania Alborghetti Apache Cassandra Software Engineer |+852 6114 9265| stefania.alborghe...@datastax.com [image: cassandrasummit.org/Email_Signature] <http://cassandrasummit.org/Email_Signature>
