I want to add checkpointing to my program that reads from a set of files in a 
directory. Without checkpointing I use readFile():

              DataStream<String> text = env.readFile(
                           new TextInputFormat(new Path(inputPath)),
                           inputPath,
                          inputProcessingMode,
                          1000);

Should I use ContinuousFileMonitoringFunction / ContinuousFileReaderOperator to 
add checkpointing? Or is there an easier way?

How do I go from splits (that ContinuousFileMonitoringFunction provides) to 
actual strings? I'm not clear how ContinuousFileReaderOperator can be used.

              DataStreamSource<TimestampedFileInputSplit> split = env.addSource(
                           new ContinuousFileMonitoringFunction<String>(
                                         new TextInputFormat(new 
Path(inputPath)),
                                         inputProcessingMode,
                                         1,
                                         1000)
              );

Thanks,
Alex

Reply via email to