Re: The right approach to store intermediate value

2017-11-14 Thread Aljoscha Krettek
Hi, You should use the State API [1] for this if you want your program to be fault-tolerant. Flink will checkpoint state and make sure that it is restored in case of failures. [1] https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/state.html

The right approach to store intermediate value

2017-11-13 Thread Shengnan YU
Dear Flink committers: I am starting to learn implementing some data streaming algorithm based on Flink API. Now I am working on MisraGries algorithm. It keeps k counters to record the data summary by increment or decrement. I want to know what is the best approach to store those counters(a hash