In general, I like Bruno's proposal. Was also checking how Flink is
handling this case, and similar to Bruno's proposal, they have a
so-called "side output":
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/windows.html#getting-late-data-as-a-side-output
The only "concer
Hi Igor,
Thank you for your answers!
I think I understand your reasoning.
Looking at your proposal, some questions/comments arose.
1. Who would be responsible for the topic to which the late records
should be sent? Is it a topic users create and manage or is it an
internal topic that is mana
Hi Matthias, Bruno,
Let me elaborate on my suggestions regarding late-tick handling.
> why you rejected the more general solution involving a callback
The main reasons why I tend to the topic approach is API-consistency and
cleanness of the user code.
The problem with a callback, in my opinion,
Hi Igor,
Thanks for the KIP!
Similar to Matthias, I am also wondering why you rejected the more
general solution involving a callback. I also think that writing to a
topic is just one of multiple ways to handle late records. For example,
one could compute statistics over the late records befo
Thanks for the KIP Igor.
What you propose sounds a little bit like a "dead-letter-queue" pattern.
Thus, I am wondering if we should try to do a built-in
"dead-letter-queue" feature that would be general purpose? For example,
uses can drop message in the source node if they don't have a valid
times
