Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/3535#discussion_r105956185
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/windowing/WindowOperator.java
---
@@ -354,22 +354,27 @@ public void merge(W mergeResult,
}
});
- // drop if the window is already late
- if (isLate(actualWindow)) {
-
mergingWindows.retireWindow(actualWindow);
- continue;
- }
+ context.key = key;
+ context.window = actualWindow;
W stateWindow =
mergingWindows.getStateWindow(actualWindow);
if (stateWindow == null) {
throw new IllegalStateException("Window
" + window + " is not in in-flight window set.");
}
windowState.setCurrentNamespace(stateWindow);
- windowState.add(element.getValue());
- context.key = key;
- context.window = actualWindow;
+ // Drop if the window is already late. In rare
cases (with a misbehaving
+ // WindowAssigner) it can happen that a window
becomes late that already has
+ // state (contents, state and timers). That's
why we first get the window state
+ // above and then drop everything.
+ if (isLate(actualWindow)) {
+ clearAllState(actualWindow,
windowState, mergingWindows);
+ mergingWindows.persist();
--- End diff --
The null check is not needed here since since we know from the if block
we're in (`if (windowAssigner instanceof MergingWindowAssigner)`) that we do in
fact have merging windows.
Now that I look at it, though, I realise that the
`mergingWindows.persist()` call is not necessary because we already call it at
the end of the if block. So thanks for making me notice! ð
I moved the call out of `clearAllState()` in the first place because all
places where `clearAllState()` are called already persist afterwards. See, for
example, `onEventTime()` and `onProcessingTime()`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
