Re: Calculating stateful counts per key

2016-12-20 Thread Fabian Hueske
Hi Mäki, some additions to Greg's answer: The flatMapWithState shortcut of the Scala API uses Flink's key-value state while your TestCounters class uses the Checkpointed interface. As Greg said, the checkpointed interface operates on an operator level not per key. The key-value state automatically

Re: Calculating stateful counts per key

2016-12-20 Thread Greg Hogan
Hi Mäki, This is the expected output. Your RichFlatMapFunction is opened once per task and you are sharing counterValue for all keys processed by that task. Greg On Mon, Dec 19, 2016 at 11:38 AM, Mäki Hanna wrote: > Hi, > > > > I'm trying to calculate stateful counts per key with checkpoints

Calculating stateful counts per key

2016-12-19 Thread Mäki Hanna
Hi, I'm trying to calculate stateful counts per key with checkpoints following the example in https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/streaming/state.html#checkpointing-instance-fields. I would expect my test program to calculate the counts per key, but it seems to gro