Hi, I have a job where we read data from either Kafka or a file (for testing), decode the entries and flat map them into events, and then add a timestamp and watermark assigner to the events in a later operation. This seems to generate periodic watermarks when running from a file, but when Kafka is the source we barely get any watermark updates. What could be causing this? (the environment has setAutowatermarkInterval(1000))
Do we need to do all the timestamp and watermark assignment in the Kafka source? or should it work to do it in later operations? The events do seem to get propagated through the pipeline, we're just not getting watermarks... Thanks, William
