Comment in source code of CoGroupedStreams

2021-01-21 Thread sudranga
Is this comment in the file flink-streaming-java/src/main/java/org/apache/flink/streaming/api/datastream/CoGroupedStreams.java accurate? " * Note: Right now, the groups are being built in memory so you need to ensure that they don't * get too big. Otherwise the JVM might crash." Looking at the s

Rocksdb - Incremental vs full checkpoints

2020-10-13 Thread sudranga
Hi, I have an event-window pipeline which handles a fixed number of messages per second for a fixed number of keys. When i have rocksdb as the state backend with incremental checkpoints, i see the delta checkpoint size constantly increase. Please see