I have a process function defined with these interfaces public class MyProcessFunction<IN> extends ProcessFunction<IN, OUT> implements CheckpointedFunction, ProcessingTimeCallback {...}
In snapshotState() method, I want to close files and emit the metadata about the closed files to downstream operator. it doesn't seem possible with *snapshotState(FunctionSnapshotContext context*) interface. I can keep metadata in snapshot and restore them during recovery. but if there is no input record coming for a long time, * processElement(T value, Context ctx, Collector<DataFile> out)* won't be called. Then I can't forward the restored data to downstream operator with guaranteed latency. I can add a timer. but it doesn't seem that *onProcessingTime(long timestamp)* allows me to forward output to downstream operator either. Thanks, Steven