Re: Issues with Event Time and Kafka

What Robert said is correct. However, that behaviour depends on the Trigger. You can write your own Trigger that behaves differently when late data arrives, that is, you could write a trigger that never fires for late data. In that case, you can also simply set the allowed lateness to zero, however

Re: Issues with Event Time and Kafka

th Event Time and Kafka, click here. >> NAML >> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.Na

Re: Issues with Event Time and Kafka

Thanks for the suggestion, we can definitely try that out. My one concern there is that events technically can lag for days or even months in some cases, but we only care about including the events that lag for 30 minutes or so, and would like the further lagging events to be ignored - I just want

Re: Issues with Event Time and Kafka

Hi Ethan, I believe then it is because the Watermark and Timestamps in your implementation are uncorrelated. What Watermark really is a marker that says there will be no elements with timestamp smaller than the value of this watermark. For more info on the concept see [1]

Re: Issues with Event Time and Kafka

Hi Dawid, I'm working with Max on the project Our code for the TimestampAndWatermarkAssigner is: ``` class TimestampAndWatermarkAssigner(val maxLateness: Long) extends AssignerWithPeriodicWatermarks[Row] { override def extractTimestamp(element: Row, previousElementTimestamp: Long): Long = {

Re: Issues with Event Time and Kafka

com/Issues-with-Event-Time-and-Kafka-tp12061p12084.html >> To unsubscribe from Issues with Event Time and Kafka, click here. >> NAML >> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&a

Re: Issues with Event Time and Kafka

Hi Stephan, The right number of events seem to leave the source and enter the windows, but it shows that 0 exit the windows. Also I have tried 30 minutes and not setting the watermark interval, I am not sure what I am supposed to put there the docs seem vague about that. Best, Max On Tue, Mar

Re: Issues with Event Time and Kafka

Hi! At a first glance, your code looks correct to assign the Watermarks. What is your watermark interval in the config? Can you check with the Flink metrics (if you are using 1.2) to see how many rows leave the source, how many enter/leave the window operators, etc? That should help figuring out