I think we can start a new discussion about the modification. The modification method can be done better. we automatically calculate the blockCacheSize through the configuration started by bin/bookkeeper, and then update conf/entry_location_rocksdb.conf, or I also mention a pr to make a scheme comparison ? Or make this function a switch to choose the implementation
On 2022/10/09 15:03:01 Hang Chen wrote: > I found the PR https://github.com/apache/bookkeeper/pull/3056 has > changed the rocksDB default cache size from 10% of direct memory to > 206150041(196MB), which will lead to entry read performance decrease > when there are huge number of entries stored in the ledger directory. > It will have a huge impact on the read performance when the BookKeeper > cluster upgrades from 4.14.x to 4.15.x. > > The motivation of PR 3056 is to change RocksDB configuration to a > single configuration file, and it introduces the following drawbacks. > 1. Most users, especially those unfamiliar with RocksDB, will be > confused about tuning RocksDB performance due to lack of guidance on > some important key parameters. > 2. The RocksDB blockCacheSize configuration can only be set to a fixed > value, not a percentage of direct memory. > > In order to simplify the RocksDB configuration and make it easy to > tune the performance of the RocksDB, I prefer to separate the RocksDB > configuration into two parts. > 1. The most important configurations, which is usually changed to tune > RocksDB performance, will be located in conf/bk_server.conf > 2. Other advanced configuration will be in a separate RocksDB configuration > file > > For the default RocksDB blockCacheSize change, Do you need to trigger > a new release to change it back? Current Pulsar master branch uses > BookKeeper 4.14.x, and we can change the default value back to 10% of > direct memory to avoid the entry read performance degradation caused > by upgrading Pulsar's BookKeeper dependency to 4.15.x. > > Do you guys have any suggestions? > > Thanks, > Hang >
