Sorry for the earlier incomplete email, E-mail editor issue.
I have modified the WordCounter example program to use a NiFi DataStream as
input. I would like to add checkpointing to the program to learn how it works.
I have googled and added many of the checkpointing configuration values to the
StreamExecutionEnvironment but I'm missing something critical to getting this
to work or I'm misinterpreting how the checkpointing should work. My goal is
to save state so that if the program is stopped and then restarted the counts
will continue from the last checkpoint.
The counts variable is populated by
DataStream<WordCounter> counts = text .flatMap(new Tokenizer())
.keyBy("word") .reduce(new ReduceFunction<WordCounter>() { public
WordCounter reduce(WordCounter a, WordCounter b) throws Exception {
return new WordCounter(a.getWord, a.getCount() + b.getCount()); } })
text is just a DataStream<String> populated from NiFi where lines are broken
up into individual strings.
WordCounter is a simple pojo that stores the word and a count with get/set
methods.
Please let me know if you have any questions about the code.
On Friday, April 21, 2017 11:48 AM, Blake Essing <[email protected]> wrote:
I have modified the WordCounter example program to use a NiFi DataStream as
input. I would like to add checkpointing to the program to learn how it works.
I have googled and added many of the checkpointing configuration values to the
StreamExecutionEnvironment but I'm missing something critical to getting this
to work or I'm misinterpreting how the checkpointing should work. My goal is
to save state so that if the program is stopped and then restarted the counts
will continue from the last checkpoint. The counts variable is populated by
