Re: Sensor's Data Aggregation Out of order with EventTime.

2016-10-06 Thread Maximilian Michels
This is a common problem in Event Time which is referred to as late data. You can a) change the Watermark generation code 2) Allow elements to be late and re-trigger a window execution. For 2) see https://ci.apache.org/projects/flink/flink-docs-master/dev/windows.html#dealing-with-late-data -Max

Sensor's Data Aggregation Out of order with EventTime.

2016-10-06 Thread Steve
Hi all, We have some sensors that sends data into kafka. Each kafka partition have a set of deferent sensor writing data in it. We consume the data from flink. We want to SUM up the values in half an hour intervals in eventTime(extract from data). The result is a keyed stream by sensor_id with ti