Hi Dom, There are sync phase and async phase in checkpointing. When a operator receives a barrier, it performs snapshot aka the sync phase. And when the barriers pass through all the operators including sinks, the operators will get a notification, after which they do the async part, like committing the Kafka offsets. WRT your question, the offsets would only be committed when the whole checkpoint is successfully finished. For more information, you can refer to this post[1].
[1] https://flink.apache.org/features/2018/03/01/end-to-end-exactly-once-apache-flink.html <https://flink.apache.org/features/2018/03/01/end-to-end-exactly-once-apache-flink.html> Best, Paul Lam > 在 2019年9月9日，07:06，Dominik Wosiński <wos...@gmail.com> 写道： > > Okay, thanks for clarifying. I have some followup question here. If we > consider Kafka offsets commits, this basically means that > the offsets committed during the checkpoint are not necessarily the > offsets that were really processed by the pipeline and written to sink ? I > mean If there is a window in the pipeline, then the records are saved in > the window state if the window was not emitted yet, but they are considered > as processed, thus will not be replayed in case of restart, because Flink > considers them as processed when committing offsets. Am I correct ? > > Best, > Dom.
