In my load tests, I've found FIFO compaction to offer the best performance as my job needs state only for so long. However, this particular statement in RocksDB documentation concerns me:
"Since we never rewrite the key-value pair, we also don't ever apply the compaction filter on the keys." This is from - https://github.com/facebook/rocksdb/wiki/FIFO-compaction-style I've observed that SST files are getting compacted into larger SST files until a configured threshold is reached. Thus I'm not sure what's going on anymore. My questions at this stage are: 1. If there's a value that I get from RocksDB and I decide to update this value back then will it work with FIFO compaction? 2. SST files get created each time a checkpoint is triggered. At this point, does the data for a given key get merged in case the initial data was read from an SST file while the update must have happened in memory? 3. If the answer to above is yes, then I suppose I can use FIFO compaction. However, then my question is whether the RocksDB documentation is wrong or whether Flink is doing something in addition to what RocksDB does. Thanks!
