On 08.04.20 20:14, Seth Wiesman wrote:
There is a limitation in RocksDB's JNI bridge that will cause applications to fail if list state exceeds 2GB. I am not aware of anyone working on this issue.
That is correct, and here's the Jira issue for it: https://issues.apache.org/jira/browse/FLINK-6761
There's also another issue about changing RocksDB list state to not store the whole ListState in a single RocksDB value but instead store the individual ListState entries in individual RocksDB values, under a specially formed "sequential" key. This is the issue: https://issues.apache.org/jira/browse/FLINK-8297
Best, Aljoscha
