I want to add checkpointing to my program that reads from a set of files in a directory. Without checkpointing I use readFile():
DataStream<String> text = env.readFile( new TextInputFormat(new Path(inputPath)), inputPath, inputProcessingMode, 1000); Should I use ContinuousFileMonitoringFunction / ContinuousFileReaderOperator to add checkpointing? Or is there an easier way? How do I go from splits (that ContinuousFileMonitoringFunction provides) to actual strings? I'm not clear how ContinuousFileReaderOperator can be used. DataStreamSource<TimestampedFileInputSplit> split = env.addSource( new ContinuousFileMonitoringFunction<String>( new TextInputFormat(new Path(inputPath)), inputProcessingMode, 1, 1000) ); Thanks, Alex