In my load tests, I've found FIFO compaction to offer the best performance
as my job needs state only for so long. However, this particular statement
in RocksDB documentation concerns me:

"Since we never rewrite the key-value pair, we also don't ever apply the
compaction filter on the keys."

This is from -
https://github.com/facebook/rocksdb/wiki/FIFO-compaction-style

I've observed that SST files are getting compacted into larger SST files
until a configured threshold is reached. Thus I'm not sure what's going on
anymore.

My questions at this stage are:

   1. If there's a value that I get from RocksDB and I decide to update
   this value back then will it work with FIFO compaction?
   2. SST files get created each time a checkpoint is triggered. At
   this point, does the data for a given key get merged in case the initial
   data was read from an SST file while the update must have happened in
   memory?
   3. If the answer to above is yes, then I suppose I can use FIFO
   compaction. However, then my question is whether the RocksDB documentation
   is wrong or whether Flink is doing something in addition to what RocksDB
   does.

Thanks!

Reply via email to