Hi, You should use the State API [1] for this if you want your program to be fault-tolerant. Flink will checkpoint state and make sure that it is restored in case of failures.
[1] https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/state.html <https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/state.html> Best, Aljoscha > On 14. Nov 2017, at 05:53, Shengnan YU <sy...@connect.ust.hk> wrote: > > Dear Flink committers: > I am starting to learn implementing some data streaming algorithm based on > Flink API. Now I am working on MisraGries algorithm. It keeps k counters to > record the data summary by increment or decrement. I want to know what is the > best approach to store those counters(a hashmap data structure maybe) when > using DataStream API to implement the algorithm. Now I just declared a map > variable in the operator, is it right or I need to use some other features > like state. This may be a very basic question forgive me since I am just a > beginnerT^T. > > Best regards > ------------------------------------------------------ > Shengnan YU(於圣楠） > > sy...@connect.ust.hk<mailto:sy...@connect.ust.hk> > +852 69963058 >
