dosubot[bot] commented on issue #3059: URL: https://github.com/apache/hugegraph/issues/3059#issuecomment-4679589186
<!-- Answer --> HugeGraph's memory management in RocksDB mode involves several layers, and high memory that doesn't get reclaimed is a known challenge. Here's what's happening and what you can do: **Why memory stays high:** 1. **Off-heap memory fragmentation**: RocksDB allocates memtables and block cache off-heap. The default glibc `ptmalloc` allocator fragments memory and prevents arena reclamation [[1]](https://github.com/apache/hugegraph/discussions/2268). A recommended mitigation is to replace it with **jemalloc or tcmalloc** via `LD_PRELOAD` [[1]](https://github.com/apache/hugegraph/discussions/2268). 2. **Transaction listener leak**: A known memory leak exists in `CachedGraphTransaction` where orphaned store event listeners accumulate over time when graphs are repeatedly created/closed [[2]](https://github.com/apache/hugegraph/issues/3033). This causes slow but persistent heap growth. 3. **Vertex/edge caching**: Large Gremlin queries can cause all results to be cached in heap memory. With `vertex.cache_capacity` and `edge.cache_capacity` set high, memory won't be freed until the cache expires or evicts entries [[3]](https://github.com/apache/hugegraph/issues/793). **Key RocksDB memory configuration options to tune** [[4]](https://github.com/apache/hugegraph/blob/9126c80e414c073b60fc074361b211cfb409e0eb/hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBOptions.java): | Setting | Default | Recommendation | |---------|---------|---------------| | `rocksdb.block_cache_capacity` | 8MB | Set to 30-50% of available RAM for your workload | | `rocksdb.write_buffer_size` | 128MB | Reduce if memory constrained | | `rocksdb.max_write_buffer_number` | 6 | Reduce to 3-4 to limit memtable memory | | `rocksdb.total_memory_size` | 48GB | Set to your actual available off-heap budget | | `rocksdb.write_buffer_ratio` | 0.66 | Adjust based on read vs write workload | **Actionable steps for your 32GB RAM environment:** 1. **Use jemalloc**: Add `LD_PRELOAD=/path/to/libjemalloc.so` to your startup script to prevent memory fragmentation [[1]](https://github.com/apache/hugegraph/discussions/2268). 2. **Reduce `rocksdb.total_memory_size`**: The default is 48GB which exceeds your system RAM. Set it to something reasonable like 8-12GB given your 32GB machine also needs heap and OS memory [[5]](https://github.com/apache/hugegraph/blob/9126c80e414c073b60fc074361b211cfb409e0eb/hugegraph-store/hg-store-rocksdb/src/main/java/org/apache/hugegraph/rocksdb/access/RocksDBOptions.java). 3. **Tune caches**: Lower `vertex.cache_capacity` and `edge.cache_capacity` and set a reasonable `cache_expire` value [[6]](https://github.com/apache/hugegraph/issues/2204). 4. **JVM heap**: Ensure your `-Xmx` isn't too large—leave room for RocksDB off-heap allocations. For 32GB total RAM with RocksDB, something like `-Xms4g -Xmx8g` is reasonable, depending on your query patterns. 5. **Trigger compaction**: After large data deletions or graph drops, call `compactRange()` via the API to reclaim SSTable space and reduce memory pinned by obsolete data [[7]](https://github.com/apache/hugegraph/blob/9126c80e414c073b60fc074361b211cfb409e0eb/hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBStdSessions.java). Regarding the InterruptedException/Timeout after large queries—this is likely caused by GC pressure or RocksDB write stalls when memory is exhausted. The tuning above should help, but you may also want to monitor via `/metrics/backend` and `/metrics/system` endpoints (note: these can underreport actual RocksDB memory usage by a significant margin) [[8]](https://github.com/apache/hugegraph/issues/602). <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=reply-with-mention&utm_term=apache%2Fhugegraph).* --- Share context across your team and agents. Try [Dosu](https://dosu.dev?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-tagline&utm_term=apache%2Fhugegraph). [](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-feedback&utm_term=apache%2Fhugegraph&message_id=c0d99d43-2e4c-4a95-bc96-9df1c4e89246) [](https://github.dosu.com/apache/hugegraph?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-learn-repo&utm_term=apache%2Fhugegraph) [](https://app.dosu.dev/signup?referrer=openSource&source=github-footer&utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-add-team&utm_term=apache%2Fhugegraph) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
