Re: Custom Watermarks with Flink

2018-06-25 Thread Fabian Hueske
Hi, I would not encode this information in watermarks. Watermarks are rather an internal mechanism to reason about event-time. Flink also generates watermarks internally. This makes the behavior less predictive. You could either inject special meta data records (which Flink handles just like othe

Custom Watermarks with Flink

2018-06-23 Thread Wyatt Frelot
I originally posted to Stack Overflow because I was trying to figure out he to do this in Flink. Wondering how to implement something of the sort: (1) *write up: * https://drive.google.com/file/d/0Bw69DO1tid2_SzVVendtUV9WMVdIUXptQ1hHSl9KNjAyMTBn/view?usp=drivesdk (2) *original post: * https://sta