Hello, I'm totally new to Flink, and I'd like to make sure I understand things properly around watermarks.
We're processing messages from iot devices. Those messages have a timestamp, and we have a first phase of processing based on this timestamp. So far so good. These messages actually "pack" together several measures taken at different times, typically going from ~15mn back in the past from the message timestamp, to a few seconds back. So at a point in the processing, I'll flatMap the message stream into a stream of measures, and I'll first need to reaffect the event time. I guess I can do it using a TimestampAssigner, correct ? The flatmapped stream will now mix together a large range of event-times (so, a span of 15mn). What should I do regarding the watermark ? Should I regenerate one ? and how ? My measures will go through windowed aggregations. Should I use the allowedLateness param to manage that properly ? (Note: I'm ok with windows firing several times with updated content, if that matters. Our downstream usage is made for that.) Thanks a lot for your insights and pointers :-) Mathieu
