Hi guys, More than one year ago, we had a discussion[1] about reverting the RocksDB version for branch-4.14, and the main concern is that RocksDB 6.17.3 breaks runtime compatibility with older versions and it requires the code to be recompiled. Lari did a great investigation and you can find more details in this Apache Pulsar pull https://github.com/apache/pulsar/pull/14962. In the discussion, we have two solutions, and we choose solution 1 in the end. 1) Do not upgrade RocksDB in BookKeeper 4.14.5 and give the user which uses RocksDB an easy path to upgrade from BK 4.14.4 to 4.14.5 2) Upgrade RocksDB and requires to all the BK users to pay attention to the RocksDB version they have in the classpath. The risk is to have runtime errors which will require them to rebuild the application and redeploy their services.
However, we found two issues about the current RocksDB version (6.16.4) on branch-4.14 - In our production environment, we found both RocksDB 6.10.2 and 6.16.4 has native memory leak issue[2] and after upgraded the RocksDB version to 6.29.4.1, the memory leak issue was fixed. - We have upgraded the RocksDB version to 7.9.2 since 4.16.0, and the RocksDB can only rollback to 6.27+. It means BookKeeper 4.16.0+ can rollback to 4.15.x (RocksDB 6.29.4.1) but can't rollback to 4.14.x (RocksDB 6.16.4) if we don't upgrade the RocksDB version to 6.29.4.1 for branch-4.14 The main reason for rolling back RocksDB version for branch-4.14 is that RocksDB 6.17.3 breaks runtime compatibility with older versions and it requires the code to be recompiled. In my opinion, if users upgrade BookKeeper 4.14.7 to 4.14.8, which upgraded the RocksDB version from 6.16.4 to 6.29.4.1, they need to use the new BookKeeper package which has been compiled with new RocksDB version instead of replace the BookKeeper-server library directly. So it is safe to upgrade the RocksDB version. I have pushed one PR[4] to upgrade the RocksDB version to 6.29.4.1 for branch-4.14. Any concerns? Thanks, Hang [1] https://lists.apache.org/thread/dsdltmoy5ggqx4oj7hrt13hqok80d4jf [2] https://github.com/apache/bookkeeper/issues/3507 [3] https://github.com/apache/bookkeeper/issues/3734#issuecomment-1407626941 [4] https://github.com/apache/bookkeeper/pull/3947
