A. Sophie Blee-Goldman created KAFKA-12748:
----------------------------------------------
Summary: Explore new RocksDB options to consider enabling by
default
Key: KAFKA-12748
URL: https://issues.apache.org/jira/browse/KAFKA-12748
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: A. Sophie Blee-Goldman
With the rocksdb version bump comes a lot of new options, some of which look
interesting enough to explore for usage in Streams. We should try setting these
as default options and run the benchmarks to look for any performance benefit
(or decrease). See javadocs for all Options
[here|https://javadoc.io/doc/org.rocksdb/rocksdbjni/latest/org/rocksdb/Options.html]
Options.setAvoidUnnecessaryBlockingIO:
- As the name suggest, avoids blocking/long-latency tasks by scheduling a
background job to do it
Options.setBestEffortsRecovery:
- Interesting feature to allow recovering missing files without the use of
the WAL. Could be useful if the on-disk state is corrupted (eg user deletes a
file) without needing to rebuild state from scratch. Though I'd want to dig in
further to understand what exactly it does and does not do. Not a performance
improvement but we should run the benchmarks to make sure it doesn't make the
performance worse.
Options.setWriteDbidToManifest:
- Should be set to true if/when we ever need to rely on the DB id eg for
backups. Also not a performance improvement but we should still benchmark this.
Options.optimizeForSmallDb:
- This one is definitely not something we should set by default, as "small"
here means under 1GB. But it's probably worth at least calling out in the docs
for those users who know their data set size (per store) is under a GB
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
