The GitHub Actions job "Flink CI (beta)" on flink.git/release-2.0 has failed.
Run started by GitHub user AHeise (triggered by AHeise).

Head commit for run:
645a0734b33b159a6169cb12c49194aefb9b017f / Arvid Heise <ar...@apache.org>
[FLINK-37605][runtime] Infer checkpoint id on endInput in sink

So far, we used a special value for the final checkpoint on endInput. However, 
as shown in the description of this ticket, final doesn't mean final. Hence, 
multiple committables with EOI could be created at different times.

With this commit, we stop using a special value for such committables and 
instead try to guess the checkpoint id of the next checkpoint. There are 
various factors that influence the checkpoint id but we can mostly ignore them 
all because we just need to pick a checkpoint id that is
- higher than all checkpoint ids of the previous, successful checkpoints of 
this attempt
- higher than the checkpoint id of the restored checkpoint
- lower than any future checkpoint id.

Hence, we just remember the last observed checkpoint id (initialized with 
max(0, restored id)), and use last id + 1 for endInput. Naturally, multiple 
endInput calls happening through restarts will result in unique checkpoint ids. 
Note that aborted checkpoints before endInput may result in diverged checkpoint 
ids across subtasks. However, each of the id satisfies above requirements and 
any id of endInput1 will be smaller than any id of endInput2. Thus, diverged 
checkpoint ids will not impact correctness at all.

(cherry picked from commit 93025452714570a4d461519510375dd72af3a2c0)

Report URL: https://github.com/apache/flink/actions/runs/14503129784

With regards,
GitHub Actions via GitBox

Reply via email to