Hi Stephan, I just wrote an answer to your SO question.
Best, Fabian 2016-11-10 11:01 GMT+01:00 Stephan Epping <stephan.epp...@zweitag.de>: > Hello, > > I found this question in the Nabble archive (http://apache-flink-user- > mailing-list-archive.2336050.n4.nabble.com/Maintaining- > watermarks-per-key-instead-of-per-operator-instance-tp7288.html) but was > unable/dont know how to reply. > > Here is my question regarding the mentioned thread: > > Hello, > > I have similar requirements (see StackOverflor http:// > stackoverflow.com/questions/40465335/apache-flink- > multiple-window-aggregations-and-late-data). I am pretty new to flink, > could you elaborate on a possible solution? We can guarantee good ordering > by sensor_id, thus watermarking by key would be the only reasonable way for > us (*sensorData.keyBy('id').timeWindow(1.minute).sum('value')*), could I > do my own watermarking after > *sensorData.keyBy('id').overwriteWatermarking()*... per key? Or maybe > using custom state plus a custom trigger? What happens if a sensor dies or > is being removed completely, how can this be detected as watermarks would > be ignored for window garbage collection. Or could we dynamically schedule > a job of each sensor? Which would result in 1000 Jobs. > > > Thanks, > Stephan > > >
