Il Lun 10 Ott 2022, 07:44 Yong Zhang <zhangyong1025...@gmail.com> ha scritto:
> We shouldn't remove any existing configuration or feature in a so short > time. I think at least we need to back-support the existing configuration. > If we must do the break, we should make the existing things in a > deprecation > state for a while. > > Breaking existing configurations would make a huge impact. > Maybe we can add back the previous configurations in the next minor > release for 4.15. So user can upgrade their cluster to 4.15.3 without > changing any configuration. > > WDYT? > Great idea Enrico > Yong > > > > > On Sun, 9 Oct 2022 at 23:20, Enrico Olivelli <eolive...@gmail.com> wrote: > > > Il Dom 9 Ott 2022, 17:03 Hang Chen <chenh...@apache.org> ha scritto: > > > > > I found the PR https://github.com/apache/bookkeeper/pull/3056 has > > > changed the rocksDB default cache size from 10% of direct memory to > > > 206150041(196MB), which will lead to entry read performance decrease > > > when there are huge number of entries stored in the ledger directory. > > > It will have a huge impact on the read performance when the BookKeeper > > > cluster upgrades from 4.14.x to 4.15.x. > > > > > > > > > I think that that patch shoud had passed from the BP process. > > It is a huge breaking change. > > Personally I missed it. > > > > BTW now it is there, we should have talked about it in the release notes, > > and we now must add some guidance to users upgrading to this version of > BK. > > > > It is a pity that we cannot keep the automatic value. > > BTW in all the Pulsar cluster I know I think that they are overriding > that > > value because the default value is not big enough. > > > > I think that at this point we only have to document on the website the > > breaking change and add a page about BK and RocksDB > > > > Enrico > > > > > > > The motivation of PR 3056 is to change RocksDB configuration to a > > > single configuration file, and it introduces the following drawbacks. > > > 1. Most users, especially those unfamiliar with RocksDB, will be > > > confused about tuning RocksDB performance due to lack of guidance on > > > some important key parameters. > > > 2. The RocksDB blockCacheSize configuration can only be set to a fixed > > > value, not a percentage of direct memory. > > > > > > In order to simplify the RocksDB configuration and make it easy to > > > tune the performance of the RocksDB, I prefer to separate the RocksDB > > > configuration into two parts. > > > 1. The most important configurations, which is usually changed to tune > > > RocksDB performance, will be located in conf/bk_server.conf > > > 2. Other advanced configuration will be in a separate RocksDB > > > configuration file > > > > > > For the default RocksDB blockCacheSize change, Do you need to trigger > > > a new release to change it back? Current Pulsar master branch uses > > > BookKeeper 4.14.x, and we can change the default value back to 10% of > > > direct memory to avoid the entry read performance degradation caused > > > by upgrading Pulsar's BookKeeper dependency to 4.15.x. > > > > > > Do you guys have any suggestions? > > > > > > Thanks, > > > Hang > > > > > >
