Yes, metrics from IoT is my case now.In additional to unsync clocks I also have devices that could buffer data when offline and resend it later when become online and that data also must be processed in common pipeline. But now it will be marked as 'late' and will be drop. I do some workarounds, bu
Thanks for the insightful sharing!
Best,
Zhanghao Chen
From: Lasse Nedergaard
Sent: Thursday, May 15, 2025 13:10
To: Zhanghao Chen
Cc: mosin...@yandex.ru ; user@flink.apache.org
Subject: Re: Keyed watermarks: A fine-grained watermark generation for Apache
Hi
I have talked with the community about this for many years last time at Flink
forward 2024 in Berlin.
The use case are simple. If you receive data from IoT devices over the gsm
network. The clock on all the devices aren’t synchronised the IoT devices can
buffer data to reduce the cost for
Thanks for sharing! It is an interesting idea. The generalized watermark [1]
introduced in DataStreamV2 might be sufficient to implement it. It'll be great
if you could share more contexts on why this is useful in your pipelines.
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-467%3A
