The GitHub Actions job "Nightly (beta)" on flink.git/master has failed.
Run started by GitHub user github-actions[bot] (triggered by 
github-actions[bot]).

Head commit for run:
93025452714570a4d461519510375dd72af3a2c0 / Arvid Heise <ar...@apache.org>
[FLINK-37605][runtime] Infer checkpoint id on endInput in sink

So far, we used a special value for the final checkpoint on endInput. However, 
as shown in the description of this ticket, final doesn't mean final. Hence, 
multiple committables with EOI could be created at different times.

With this commit, we stop using a special value for such committables and 
instead try to guess the checkpoint id of the next checkpoint. There are 
various factors that influence the checkpoint id but we can mostly ignore them 
all because we just need to pick a checkpoint id that is
- higher than all checkpoint ids of the previous, successful checkpoints of 
this attempt
- higher than the checkpoint id of the restored checkpoint
- lower than any future checkpoint id.

Hence, we just remember the last observed checkpoint id (initialized with 
max(0, restored id)), and use last id + 1 for endInput. Naturally, multiple 
endInput calls happening through restarts will result in unique checkpoint ids. 
Note that aborted checkpoints before endInput may result in diverged checkpoint 
ids across subtasks. However, each of the id satisfies above requirements and 
any id of endInput1 will be smaller than any id of endInput2. Thus, diverged 
checkpoint ids will not impact correctness at all.

Report URL: https://github.com/apache/flink/actions/runs/14459924767

With regards,
GitHub Actions via GitBox

Reply via email to