Re: user driven stream processing

2017-11-29 Thread zanqing zhang
Thanks Fabian and Tony for the info. It's very helpful. Looks like the general approach is to implement a job topology containing parameterized (CoXXXMapFunction) operators. The user defined parameters will be ingested using the extra input the CoXXXMapFunction take. Ken On Wed, Nov 29, 2017 at

Re: user driven stream processing

2017-11-29 Thread Fabian Hueske
Another example is King's RBEA platform [1] which was built on Flink. In a nutshell, RBEA runs a single large Flink job, to which users can add queries that should be computed. Of course, the query language is restricted because they queries must match on the structure of the running job. Hope thi

Re: user driven stream processing

2017-11-28 Thread Tony Wei
Hi KZ, https://data-artisans.com/blog/real-time-fraud-detection-ing-bank-apache-flink This article seems to be a good example to trigger a new calculation on a running job. Maybe you can get some help from it. Best Regards, Tony Wei 2017-11-29 4:53 GMT+08:00 zanqing zhang : > Hi All, > > Has an

user driven stream processing

2017-11-28 Thread zanqing zhang
Hi All, Has anyone done any stream processing driven by a user request? What's the recommended way of doing this? Or is this completely wrong direction to go for applications running on top of Flink? Basically we need to tweak the stream processing based on parameters provided by a user, e.g. sho