Hi, > On 6 Nov 2018, at 18:22, PranjalChauhan <[email protected]> wrote: > > Thank you for your response Piotr. I plan to upgrade to Flink 1.5.x early > next year. > > Two follow-up questions for now. > > 1. > " When operator snapshots are taken, there are two parts: the synchronous > and the asynchronous parts. " > I understand that when the operator snapshot is being taken, the processing > of that operator is stopped as taking this snapshot is synchronous part. Is > there any other synchronous part in the snapshot / checkpoint process? >
Not as far as I know. > > 2. > Based on the test I mentioned above, my understanding is that for a window > operator, when all events that belongs to checkpoint N and the checkpoint > barrier N are received by window operator (but pending for window to be > triggered), then checkpoint barrier N will be immediately emitted to the > sink operator (so snapshot can be completed) while the events are still > pending to be evaluated by window operator. > > Can you please confirm my understanding as I was initially confused by the > following second statement (emits all pending outgoing records) under > Barriers section in this doc > https://ci.apache.org/projects/flink/flink-docs-release-1.2/internals/stream_checkpointing.html > ? > > "When an intermediate operator has received a barrier for snapshot n from > all of its input streams, it emits itself a barrier for snapshot n into all > of its outgoing streams." > > " Once the last stream has received barrier n, the operator emits all > pending outgoing records, and then emits snapshot n barriers itself. “ I think you might be mixing two different concepts, watermarks and checkpoint barriers. The documentation that you are quoting describes checkpointing mechanism, checkpoint barriers and records alignment. Checkpoint barrier do not cause any results to be emitted from WindowOperator, this happens when timers are triggered (wall clock timers in case of processing time or watermarks in case of event time). Piotrek
