dosubot[bot] commented on issue #3059: URL: https://github.com/apache/hugegraph/issues/3059#issuecomment-4679715547
<!-- Answer --> Your JVM settings look reasonable for heap, but the issue is likely that **most of the memory growth is happening off-heap** in RocksDB's native allocations (memtables, block cache, table readers), which are outside of `-Xmx` control. A few things to check and tune: **1. `rocksdb.total_memory_size` defaults to 48GB** — this alone can consume more than your 32GB machines have available. On your 32GB machines with `-Xmx16g`, you only have ~14-16GB left for OS + RocksDB native memory. Set this to something like **8-10GB** on 32GB machines [[1]](https://github.com/apache/hugegraph/blob/9126c80e414c073b60fc074361b211cfb409e0eb/hugegraph-store/hg-store-rocksdb/src/main/java/org/apache/hugegraph/rocksdb/access/RocksDBOptions.java). **2. Replace glibc's memory allocator with jemalloc.** RocksDB's allocation pattern causes severe fragmentation with the default `ptmalloc`, preventing the OS from reclaiming freed memory [[2]](https://github.com/apache/hugegraph/discussions/2268). Add to your startup: ```bash export LD_PRELOAD=/path/to/libjemalloc.so ``` This is often the single most impactful fix for the "memory never gets reclaimed" symptom. **3. Tune write buffers.** With the default `max_write_buffer_number=6` and `write_buffer_size=128MB`, each column family can hold up to ~768MB in memtables. If you have many graphs (each with multiple column families), this adds up fast [[3]](https://github.com/apache/hugegraph/blob/9126c80e414c073b60fc074361b211cfb409e0eb/hugegraph-server/hugegraph-rocksdb/src/main/java/org/apache/hugegraph/backend/store/rocksdb/RocksDBOptions.java). Consider reducing: ```properties rocksdb.write_buffer_size=64m rocksdb.max_write_buffer_number=3 ``` **4. Known transaction listener leak.** Repeatedly creating graphs can trigger a memory leak in `CachedGraphTransaction` where orphaned event listeners accumulate and never get cleaned up [[4]](https://github.com/apache/hugegraph/issues/3033). This would explain why memory grows with each new graph creation and is never reclaimed. **5. Reduce vertex/edge cache sizes.** If you're running large Gremlin queries, the in-heap vertex and edge caches can consume significant heap. Lower `vertex.cache_capacity` and `edge.cache_capacity` and set a shorter `cache_expire` [[5]](https://github.com/apache/hugegraph/issues/793). To diagnose whether the problem is heap vs. off-heap, compare your JVM heap usage (via `jstat -gc` or GC logs) against total process RSS (via `ps` or `/proc/<pid>/status`). If heap is well under 16GB but RSS is 30GB+, the problem is squarely in RocksDB native memory and the allocator fragmentation issue. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=reply-with-mention&utm_term=apache%2Fhugegraph).* --- Share context across your team and agents. Try [Dosu](https://dosu.dev?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-tagline&utm_term=apache%2Fhugegraph). [](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-feedback&utm_term=apache%2Fhugegraph&message_id=adf4cb65-8b1c-4c91-a1ed-f63fabb1075c) [](https://github.dosu.com/apache/hugegraph?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-learn-repo&utm_term=apache%2Fhugegraph) [](https://app.dosu.dev/signup?referrer=openSource&source=github-footer&utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-add-team&utm_term=apache%2Fhugegraph) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
