Checkpointing

2016-12-14 Thread Mäki Hanna
Hi, I'm learning Flink and trying to calculate counters that are checkpointed at given intervals. Following the examples on page https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/streaming/state.html, except in Scala, I tried to create a function class CountersWithState extends

Calculating stateful counts per key

2016-12-19 Thread Mäki Hanna
Hi, I'm trying to calculate stateful counts per key with checkpoints following the example in https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/streaming/state.html#checkpointing-instance-fields. I would expect my test program to calculate the counts per key, but it seems to gro

FlinkML and DataStream API

2016-12-21 Thread Mäki Hanna
Hi, I'm wondering if there is a way to use FlinkML and make predictions continuously for test data coming from a DataStream. I know FlinkML only supports the DataSet API (batch) at the moment, but is there a way to convert a DataStream into DataSets? I'm thinking of something like (0. fit mod