Re: Window on stream with timestamps ascending by key

2016-03-24 Thread Stephan Ewen
If you would not be using watermarks at all, but only special events that signal the "per key watermark", then you can simply build this on top of "keyBy().flatMap()". The custom operator would only be needed if you need access to the watermarks, of the processing time trigger scheduler. On Thu,

Re: Window on stream with timestamps ascending by key

2016-03-24 Thread cam
Hi, I'd be happy to implement such an operator, I'm failry new to Flink so I'm still diving into it as fast as I can. Do you have some more insights about what to implement over OneInputStreamOperator ? I'm not sure if I should play with watermarks here or build a similar concept. How is that dif

Re: Window on stream with timestamps ascending by key

2016-03-19 Thread Aljoscha Krettek
Hi, what you essentially would require is watermarks that are tracked by key. Right now this is not possible in Flink. The watermarks, which are used for keeping track of the timestamps, are global across all keys. Maybe you could implement something that fits your requirements in a custom oper