MartijnVisser opened a new pull request, #28650:
URL: https://github.com/apache/flink/pull/28650
## What is the purpose of the change
Backport of #28608 to release-1.20.
An interrupted timer-firing chain defers its continuation (and the
downstream watermark emission) to a deferrable mail. The operator-finish drain
skipped deferrable mails, so `EndOfData` could overtake the deferred watermark
and downstream operators never received it, losing the state of windows that
only fire on it (FLINK-39481, observed as `WindowDistinctAggregateITCase`
exactly-once data loss after restore).
The fix executes deferrable mails when finishing an operator.
## Brief change log
Cherry-pick of 83055aead2e and bbc688cef98 from master. The production fix
applied cleanly (on release-1.20 the task/mailbox classes live under
`flink-streaming-java`, and the master-only urgent-mails context was dropped
from the `TaskMailboxImpl` hunks).
The new test needed a port to the release-1.20 API: there is no
`useInterruptibleTimers` hook and no
`execution.checkpointing.unaligned.interruptible-timers.enabled` option on this
branch; interruptible timers are operator-opt-in. The test operator therefore
follows the branch-native pattern of the neighboring operator in the same test
class (implements `YieldingOperator`, wires a `MailboxWatermarkProcessor` in
`open()`/`processWatermark`), and the 2.x-only config lines are dropped. The
ported test still exercises the fixed path: the release-1.20
`MailboxWatermarkProcessor` submits its watermark continuation with
`MailOptions.deferrable()`, which is exactly the mail the fix now drains at
operator finish.
## Verifying this change
`StreamOperatorWrapperTest`, `MailboxExecutorImplTest` and
`UnalignedCheckpointsInterruptibleTimersTest` (including the new
`testDeferredWatermarkIsEmittedBeforeEndOfData`, the red-green test for this
fix) pass on this branch. `WindowDistinctAggregateStressITCase` is included in
the cherry-pick and is validated by CI.
## Does this pull request potentially affect one of the following parts:
Same as #28608: no dependencies, no public API, no serializers, no
per-record code paths; touches the operator-finish path of the task lifecycle.
## Documentation
- Does this pull request introduce a new feature? no
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (Claude Fable 5)
Generated-by: Claude Fable 5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
