Hi,
Oh, now I understand your problem.
I dont' think that Flink is able to remove the metadata early. The
implementation is designed for the general case which needs to support the
case where the window data is not purged.
Something that might work is to not configure the window operator with
allo
Hi Fabian, Thank you for your reply. I'm not sure my question was clear enough so I'll try to explain our scenario:We are working in “event time” mode.We want to handle ‘late data’ up to last X days (for example last 7 days)For each incoming event:The event is being aggregated using window functio
Hi,
A window needs to keep the data as long as it expects new data.
This is clearly the case before the end time of the window was reached. If
my window ends at 12:30, I want to wait (at least) until 12:30 before I
remove any data, right?
In case you expect some data to be late, you can configure
Hi, I'm interested in why metadata like WindowOperator and InternalTimer are being kept for windowSize + allowedLateness period per each pane.What is the purpose of keeping this data if no new events are expected to enter the pane? Is there any way this metadata can be released earlier?
