Hi, I'm trying to debug an issue I'm having with Cassandra 3.6+ and Snappy compressed tables - CASSANDRA-11993 (that may be related to CASSANDRA-5863).
I'm probably 3% familiar with Cassandra internals, so apologies if I'm wrong here - I was looking at a change in CASSANDRA-5863 where a cache was added to the SSTableIterator code path, which may have included org.apache.cassandra.utils.memory.BufferPool into the read path. My confusion comes from here - AFAICT, when the BufferPool needs to allocate more memory, it calls allocate(int size, boolean onHeap) ( https://github.com/apache/cassandra/blob/30bb255ec9fb36ace2aab51474bd3bfb9bbd3bed/src/java/org/apache/cassandra/utils/memory/BufferPool.java#L82). The value for whether onHeap is true is constant ALLOCATE_ON_HEAP_WHEN_EXAHUSTED. ALLOCATE_ON_HEAP_WHEN_EXAHUSTED is defined by the yaml setting buffer_pool_use_heap_if_exhausted which is true by default (and was so in my install). Further still ( https://github.com/apache/cassandra/blob/30bb255ec9fb36ace2aab51474bd3bfb9bbd3bed/src/java/org/apache/cassandra/utils/memory/BufferPool.java), I can't see any logic where BufferPool will actually allocate off heap memory - it seems, due to the buffer_pool_use_heap_if_exhausted setting, always allocate memory on heap. It never checks if the off heap memory is exhausted (AFAICT from these 3 files). Sure enough, if I set buffer_pool_use_heap_if_exhausted, my issue at CASSANDRA-11993 goes away (I think? I'm having a number of issues lately and I may have traded one exception for another). Is this the correct behavior of buffer_pool_use_heap_if_exhausted? Nimi
