Hello Community, I wanted to share some work I have been doing on the frocksdb and Flink that I think is useful for the Flink community.
I have implemented support for Java base associative merge operators in https://github.com/ververica/frocksdb and use that to support additional reducing and aggregating state variables in Flink. I have used this do event reordering in a flink app. All the code is in my github repo ( https://github.com/soumitrak) and I will be more than happy to work with the members to contribute the code back to frocksdb and Flink. I have filed a followup task ( https://issues.apache.org/jira/browse/FLINK-39456) to leverage the support in frocksdb to expose the state variables in the Flink. Code in my forked repos: https://github.com/soumitrak/frocksdb/commits/FRocksDB-8.10.0-SK/ - Created a branch off FRocksDB-8.10.0 and committed the changes https://github.com/soumitrak/flink/commits/rocksdb-merge-operator/ - Created a branch off master and committed the changes https://github.com/soumitrak/flink_streaming_event_reordering - Test project used to build, test, perf test, and compare the performance of heap-based, ValueState, and new MergeState (using associative merge operator). I should have started two email threads, but they are related, so added the details in one. Looking forward to guidance on how to go about this. Best, -Soumitra.
