Sorry, I forgot to cc you guys @Fabian, @Timo, @Till, @Kostas 2017-06-07 15:42 GMT+08:00 Jark Wu <j...@apache.org>:
> Hi devs, > > Dian and me and our teammates have investigated this for a long time. We > think consolidating Flink SQL and CEP is an exciting thing for Flink. It'll > make SQL more powerful and give users the ability to easily and quickly > build CEP applications. And I find Flink community has also talked about > this idea before, such as the mailing list [1] and [2] and Fabian & Till's > talk in Flink Forward 2016 [3]. > > I think THIS IS THE POINT to bring up this topic again. Because we already > have pattern matching foundation in Flink CEP library, and Stream SQL is > ready now and Calcite has partially supported pattern matching syntax! We > also drafted a design doc about how to integrate SQL and CEP, and how to > support CEP on Table API. https://docs.google.com/document/d/ > 1HaaO5eYI1VZjyhtVPZOi3jVzikU7iK15H0YbniTnN30/edit?usp=sharing > > > @Fabian, @Timo, @Till, @Kostas I include you into this discussion, it > would be great to hear your response. > > > What do others think? > > > [1] http://apache-flink-mailing-list-archive.1008284.n3. > nabble.com/Add-CEP-library-to-Flink-td9743.html#a9787 > > [2] http://apache-flink-mailing-list-archive.1008284.n3. > nabble.com/Effort-to-add-SQL-StreamSQL-to-Flink-td9727.html#a9790 > > [3] https://www.slideshare.net/tillrohrmann/streaming- > analytics-cep-two-sides-of-the-same-coin > > > Regards, > > Jark Wu > > > > 2017-06-07 13:50 GMT+08:00 Dian Fu <dia...@apache.org>: > >> Hi everyone, >> >> Flink's CEP library is a great library for complex event processing, more >> and more customers are expressing their interests in it. But it also has >> some limitations that users usually have to write a lot of code even for a >> very simple pattern match use case as it currently only supports the Java >> API. >> We have investigated some popular CEP products such as esper [1] and >> siddhi >> [2] and found that most of these CEP products support SQL-like expressions >> such as EPL to describe the match pattern. But these solutions also have >> the drawbacks that the pattern match languages are not standard SQL, the >> learn curve is steep for users and it's impossible to integrate them into >> the Flink Table API & SQL. >> We find that Oracle's CEP solution CQL [3] supports a new pattern >> recognition clause match_recognize which is a pattern recognition clause >> proposed in this paper [4]. It proposes a set of new syntaxes to define >> match pattern in sql expression. Calcite already supports part of this >> standard [5]. I think it will be of great value to support expressing >> pattern recognition clause with match_recognize clause by integrating it >> with Flink Table API & SQL and the Flink CEP library. Any thoughts? >> >> [1] http://www.espertech.com >> [2] https://github.com/wso2/siddhi >> [3] >> https://docs.oracle.com/middleware/1213/eventprocessing/cql- >> reference/GUID-34D4968E-C55A-4BC7-B1CE-C84B202217BD.htm#CQLLR1531 >> [4] >> http://web.cs.ucla.edu/classes/winter17/cs240B/notes/row- >> pattern-recogniton-11.pdf >> [5] https://issues.apache.org/jira/browse/CALCITE-1570 >> >> Best Regards, >> Dian >> > >
