Re: BoundedOutOfOrdernessTimestampExtractor and timestamps in the future

2016-11-03 Thread Maximilian Michels
The BoundedOutOfOrdernessTimestampExtractor is not really useful if you have outliers because you always set the Watermark to the element with the largest timestamp minus the out-of-orderness. If your data is of such nature, you will have to implement a custom Watermark extractor to deal with these

Re: BoundedOutOfOrdernessTimestampExtractor and timestamps in the future

2016-11-01 Thread Konstantin Knauf
Hi Dominik, out of curiosity, how come that you receive timestamps from the future? ;) Depending on the semantics of these future events, it might also make sense to already "floor" the timestamp to processing time in the extractTimestamp()-Method. I am not sure, if I understand your follow up q

BoundedOutOfOrdernessTimestampExtractor and timestamps in the future

2016-11-01 Thread Dominik Bruhn
Hey, I'm using a BoundedOutOfOrdernessTimestampExtractor for assigning my timestamps and discarding to old events (which happens sometimes). Now my problem is that some events, by accident have timestamps in the future. If the timestamps are more in the future than my `maxOutOfOrderness`, I'm