Hi,
managing a per-key watermark would require keeping to current watermark for
each key, for example at the sources or in a timestamp/watermark assigner.
The problem then is figuring out when you can discard that state because it
would otherwise grow indefinitely if you have an evolving key space.
There's nothing stopping me assigning timestamps and generating watermarks on
a keyed stream in the implementation and the KeyedStream API supports it. It
appears the underlying operator that gets created in
DataStream.assignTimestampsAndWatermarks() isn't key-aware and globally
tracks timestamps.
Hi Jordan,
it is not possible to generate watermarks per key. This feature has been
requested a couple of times but I think there are no plans to implement
that.
As far as I understand, the management of watermarks would be quite
expensive (maintaining several watermarks, purging watermarks of exp
