Re: [DISCUSS] BookKeeper RocksDB configuration changed default blockCacheSize since 4.15.0

2022-10-17 Thread Hang Chen
For the pr https://github.com/apache/bookkeeper/pull/3523 In BookKeeper and Pulsar's conf folder, it already has default RocksDB configuration files. In this Pr, the loading order is RocksDB Configuration file > conf/bk_server.conf, which means it will still load the RocksDB configuration file by

Re: [DISCUSS] BookKeeper RocksDB configuration changed default blockCacheSize since 4.15.0

2022-10-17 Thread Yong Zhang
Thank you, Nicolò Boschi! I have updated the website and release note to mention this. https://github.com/apache/bookkeeper/pull/3540 Yong On Mon, 17 Oct 2022 at 16:08, Nicolò Boschi wrote: > The pull has been merged and cherry-picked to branch-4.15 > > Thanks, > Nicolò Boschi > > > Il giorno

Re: [DISCUSS] BookKeeper RocksDB configuration changed default blockCacheSize since 4.15.0

2022-10-17 Thread Nicolò Boschi
The pull has been merged and cherry-picked to branch-4.15 Thanks, Nicolò Boschi Il giorno gio 13 ott 2022 alle ore 12:58 Shiji Lu ha scritto: > I think we can start a new discussion about the modification. The > modification method can be done better. > we automatically calculate the blockCach

Re: [DISCUSS] BookKeeper RocksDB configuration changed default blockCacheSize since 4.15.0

2022-10-13 Thread Shiji Lu
I think we can start a new discussion about the modification. The modification method can be done better. we automatically calculate the blockCacheSize through the configuration started by bin/bookkeeper, and then update conf/entry_location_rocksdb.conf, or I also mention a pr to make a scheme

Re: [DISCUSS] BookKeeper RocksDB configuration changed default blockCacheSize since 4.15.0

2022-10-13 Thread Enrico Olivelli
Il giorno gio 13 ott 2022 alle ore 04:35 Yong Zhang ha scritto: > > Hi, > > I have pushed a PR for fixing this issue. > https://github.com/apache/bookkeeper/pull/3523 > > As we discussed, we ignore the configuration file if they are not exists. > Then using the configurations in the bk server conf

Re: [DISCUSS] BookKeeper RocksDB configuration changed default blockCacheSize since 4.15.0

2022-10-12 Thread Yong Zhang
Hi, I have pushed a PR for fixing this issue. https://github.com/apache/bookkeeper/pull/3523 As we discussed, we ignore the configuration file if they are not exists. Then using the configurations in the bk server configuration file. The rocksDB related configuration file already include in our

Re: [DISCUSS] BookKeeper RocksDB configuration changed default blockCacheSize since 4.15.0

2022-10-10 Thread Hang Chen
I agree with Yong, we should back-support the existing configuration, otherwise, the pulsar helm chart also need to be changed when upgrade BookKeeper 4.14.x to 4.15.x Thanks, Hang Enrico Olivelli 于2022年10月10日周一 16:07写道: > > Il Lun 10 Ott 2022, 07:44 Yong Zhang ha > scritto: > > > We shouldn't

Re: [DISCUSS] BookKeeper RocksDB configuration changed default blockCacheSize since 4.15.0

2022-10-10 Thread Enrico Olivelli
Il Lun 10 Ott 2022, 07:44 Yong Zhang ha scritto: > We shouldn't remove any existing configuration or feature in a so short > time. I think at least we need to back-support the existing configuration. > If we must do the break, we should make the existing things in a > deprecation > state for a wh

Re: [DISCUSS] BookKeeper RocksDB configuration changed default blockCacheSize since 4.15.0

2022-10-09 Thread Yong Zhang
We shouldn't remove any existing configuration or feature in a so short time. I think at least we need to back-support the existing configuration. If we must do the break, we should make the existing things in a deprecation state for a while. Breaking existing configurations would make a huge impa

Re: [DISCUSS] BookKeeper RocksDB configuration changed default blockCacheSize since 4.15.0

2022-10-09 Thread Enrico Olivelli
Il Dom 9 Ott 2022, 17:03 Hang Chen ha scritto: > I found the PR https://github.com/apache/bookkeeper/pull/3056 has > changed the rocksDB default cache size from 10% of direct memory to > 206150041(196MB), which will lead to entry read performance decrease > when there are huge number of entries s

[DISCUSS] BookKeeper RocksDB configuration changed default blockCacheSize since 4.15.0

2022-10-09 Thread Hang Chen
I found the PR https://github.com/apache/bookkeeper/pull/3056 has changed the rocksDB default cache size from 10% of direct memory to 206150041(196MB), which will lead to entry read performance decrease when there are huge number of entries stored in the ledger directory. It will have a huge impact