[
https://issues.apache.org/jira/browse/IGNITE-16538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Evgeny Stanilovsky updated IGNITE-16538:
----------------------------------------
Fix Version/s: 3.2
(was: 3.1)
> Enable RocksDB to control memory usage better with stalling write if memory
> usage exceeds limit
> -----------------------------------------------------------------------------------------------
>
> Key: IGNITE-16538
> URL: https://issues.apache.org/jira/browse/IGNITE-16538
> Project: Ignite
> Issue Type: Improvement
> Reporter: Yun Tang
> Priority: Major
> Labels: iep-74, ignite-3
> Fix For: 3.2
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Currently, Ignite choose RocksDB-6.20.3 and let all RocksDB table storages
> share same {{WriteBufferManager}} within a data region. This could limit the
> memory of all storage instances in a way, however, this is still not enough
> considerring the case that current {{WriteBufferManager}} cannot not limit
> the memory even memory_usage exceeds buffer_size with multi rocksdb instances.
> We can refer to this feature descriptions:
> "When WriteBufferManager is shared across DBs and column families
> to maintain memory usage under a limit, OOMs have been observed when flush
> cannot
> finish but writes continuously insert to memtables.
> In order to avoid OOMs, when memory usage goes beyond buffer_limit_ and DBs
> tries to write,
> this change will stall incoming writers until flush is completed and
> memory_usage
> drops." (see [the PR summary|https://github.com/facebook/rocksdb/pull/7898])
> RocksDB introduce better memory management and public it to java side from
> RocksDB-6.27.3 (see
> [commit-dc00e4b120|https://github.com/facebook/rocksdb/commit/dc00e4b1206cba9995fe042ec976df54d0f6e76f])
> Thus, I suggest to bump the RocksDB version to 6.27.3 and enable this
> feature by default.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
