[
https://issues.apache.org/jira/browse/CASSANDRA-20491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Konstantinov updated CASSANDRA-20491:
--------------------------------------------
Description:
There are many places where Cassandra uses offheap memory and several methods
to do it, it would be nice to have a summary doc about this usage.
A draft version:
||Case||Off-heap memory type||Way to monitor||Related configuration||
|offheap_buffers memtables|Direct byte buffer|JMX:
MemtableOffHeapSize|memtable_offheap_space|
|offheap_object memtables|Native allocator|JMX:
MemtableOffHeapSize|memtable_offheap_space|
|Chunk cache|Native allocator|org.apache.cassandra.metrics:
type=BufferPool,scope=chunk-cache,name=UsedSize
nodetool info, “Chunk Cache” property|file_cache_enabled
file_cache_size|
|Hints| | | |
|Repair Merkle trees| | |repair_session_space|
|Bloom filters|Native allocator|JMX: BloomFilterOffHeapMemoryUsed
nodetool tablestats (“Bloom filter off heap memory used” property)
|depends on bloom_filter_fp_chance table property|
|Index summaries|Native allocator|JMX: IndexSummaryOffHeapMemoryUsed
nodetool tablestats (“Index summary off heap memory used” property)
|index_summary_capacity_in_mb|
|Compression metadata|Native allocator|JMX: CompressionMetadataOffHeapMemoryUsed
nodetool tablestats (”Compression metadata off heap memory used” property)
|Depends on chunk_length_in_kb table property|
|Direct IO commitlog | | | |
|Compression| | | |
|Netty
PooledByteBufAllocator|Unsafe.allocateMemory|io.netty.util.internal.PlatformDependent#usedDirectMemory|
|
| | | | |
| | | | |
Off-heap memory types:
||Type||Allocation logic||Monitoring||Limit||
|Direct byte buffer| ByteBuffer.allocateDirect(size)|JMX:
java.nio:name=direct,type=BufferPool|-XX:MaxDirectMemorySize|
|Native allocator|org.apache.cassandra.utils.memory.MemoryUtil#allocate
com.sun.jna.Native#malloc|nodetool info
"Off Heap Memory (MB)" row
MemtableOffHeapSize
+ BloomFilterOffHeapMemoryUsed
+ IndexSummaryOffHeapMemoryUsed
+ CompressionMetadataOffHeapMemoryUsed| |
|Unsafe.allocateMemory|Unsafe.allocateMemory| | |
was:
There are many places where Cassandra uses offheap memory and several methods
to do it, it would be nice to have a summary doc about this usage.
A draft version:
||Case||Off-heap memory type||Way to monitor||Related configuration||
|offheap_buffers memtables|Direct byte buffer| | |
|offheap_object memtables|Native allocator| | |
|Chunk cache| |org.apache.cassandra.metrics:
type=BufferPool,scope=chunk-cache,name=UsedSize|file_cache_enabled
file_cache_size|
|Hints| | | |
|Repair Merkle trees| | | |
|Bloom filters|Native allocator| | |
|Index summaries|Native allocator| | |
|Compression metadata|Native allocator| | |
|Direct IO commitlog | | | |
|Compression| | | |
|Netty PooledByteBufAllocator|Unsafe.allocateMemory| | |
| | | | |
| | | | |
Off-heap memory types:
||Type||Allocation logic||Monitoring||Limit||
|Direct byte buffer| ByteBuffer.allocateDirect(size)|JMX:
java.nio:name=direct,type=BufferPool|-XX:MaxDirectMemorySize|
|Native allocator|org.apache.cassandra.utils.memory.MemoryUtil#allocate
com.sun.jna.Native#malloc| | |
|Unsafe.allocateMemory|Unsafe.allocateMemory| | |
> Document offheap memory used by Cassandra
> -----------------------------------------
>
> Key: CASSANDRA-20491
> URL: https://issues.apache.org/jira/browse/CASSANDRA-20491
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Documentation
> Reporter: Dmitry Konstantinov
> Assignee: Dmitry Konstantinov
> Priority: Normal
>
> There are many places where Cassandra uses offheap memory and several methods
> to do it, it would be nice to have a summary doc about this usage.
> A draft version:
> ||Case||Off-heap memory type||Way to monitor||Related configuration||
> |offheap_buffers memtables|Direct byte buffer|JMX:
> MemtableOffHeapSize|memtable_offheap_space|
> |offheap_object memtables|Native allocator|JMX:
> MemtableOffHeapSize|memtable_offheap_space|
> |Chunk cache|Native allocator|org.apache.cassandra.metrics:
> type=BufferPool,scope=chunk-cache,name=UsedSize
>
> nodetool info, “Chunk Cache” property|file_cache_enabled
> file_cache_size|
> |Hints| | | |
> |Repair Merkle trees| | |repair_session_space|
> |Bloom filters|Native allocator|JMX: BloomFilterOffHeapMemoryUsed
> nodetool tablestats (“Bloom filter off heap memory used” property)
> |depends on bloom_filter_fp_chance table property|
> |Index summaries|Native allocator|JMX: IndexSummaryOffHeapMemoryUsed
> nodetool tablestats (“Index summary off heap memory used” property)
> |index_summary_capacity_in_mb|
> |Compression metadata|Native allocator|JMX:
> CompressionMetadataOffHeapMemoryUsed
> nodetool tablestats (”Compression metadata off heap memory used” property)
> |Depends on chunk_length_in_kb table property|
> |Direct IO commitlog | | | |
> |Compression| | | |
> |Netty
> PooledByteBufAllocator|Unsafe.allocateMemory|io.netty.util.internal.PlatformDependent#usedDirectMemory|
> |
> | | | | |
> | | | | |
> Off-heap memory types:
>
> ||Type||Allocation logic||Monitoring||Limit||
> |Direct byte buffer| ByteBuffer.allocateDirect(size)|JMX:
> java.nio:name=direct,type=BufferPool|-XX:MaxDirectMemorySize|
> |Native allocator|org.apache.cassandra.utils.memory.MemoryUtil#allocate
> com.sun.jna.Native#malloc|nodetool info
> "Off Heap Memory (MB)" row
> MemtableOffHeapSize
> + BloomFilterOffHeapMemoryUsed
> + IndexSummaryOffHeapMemoryUsed
> + CompressionMetadataOffHeapMemoryUsed| |
> |Unsafe.allocateMemory|Unsafe.allocateMemory| | |
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
