Thanks for the feedback! We will start the SQL effort with putting the existing (batch) Table API on top of Apache Calcite. >From there we continue to add streaming support for the Table API before we put a StreamSQL interface on top.
Consolidating the efforts with the CEP library sounds like a good idea to me. Maybe it can be nicely integrated with the streaming table API and later as well with the StreamSQL interface (the StreamSQL dialect is not defined yet). @Till: What do you think about adding CEP features to the Table API. From the CEP design doc, it looks like we need to add a pattern matching operator in addition to the window features that we need to add for streaming Table API in any case. Best, Fabian 2016-01-11 4:03 GMT+01:00 Jiangsong (Hi) <hi.jiangs...@huawei.com>: > I suggest refering to Esper EPL[1], which is a SQL-standard language > extend to offering a cluster of window, pattern matching. EPL can both > support Streaming SQL and CEP with one unified syntax. > > [1] > http://www.espertech.com/esper/release-5.2.0/esper-reference/pdf/esper_reference.pdf > (Chapter 5. EPL Reference: Clauses) > > > Regards > Song > > > -----邮件原件----- > 发件人: Chiwan Park [mailto:chiwanp...@apache.org] > 发送时间: 2016年1月11日 10:31 > 收件人: dev@flink.apache.org > 主题: Re: Effort to add SQL / StreamSQL to Flink > > We still don’t have a concensus about the streaming SQL and CEP library on > Flink. Some people want to merge these two libraries. Maybe we have to > discuss about this in mailing list. > > > On Jan 11, 2016, at 10:53 AM, Nick Dimiduk <ndimi...@gmail.com> wrote: > > > > What's the relationship between the streaming SQL proposed here and > > the CEP syntax proposed earlier in the week? > > > > On Sunday, January 10, 2016, Henry Saputra <henry.sapu...@gmail.com> > wrote: > > > >> Awesome! Thanks for the reply, Fabian. > >> > >> - Henry > >> > >> On Sunday, January 10, 2016, Fabian Hueske <fhue...@gmail.com > >> <javascript:;>> wrote: > >> > >>> Hi Henry, > >>> > >>> There is https://issues.apache.org/jira/browse/FLINK-2099 and a few > >>> subissues. > >>> I'll reorganize these and add more issues for the tasks described in > >>> the design document in the next days. > >>> > >>> Thanks, Fabian > >>> > >>> 2016-01-10 2:45 GMT+01:00 Henry Saputra <henry.sapu...@gmail.com > >> <javascript:;> > >>> <javascript:;>>: > >>> > >>>> HI Fabian, > >>>> > >>>> Have you created JIRA ticket to keep track of this new feature? > >>>> > >>>> - Henry > >>>> > >>>> On Thu, Jan 7, 2016 at 6:05 AM, Fabian Hueske <fhue...@gmail.com > >> <javascript:;> > >>> <javascript:;>> wrote: > >>>>> Hi everybody, > >>>>> > >>>>> in the last days, Timo and I refined the design document for > >>>>> adding a > >>>> SQL / > >>>>> StreamSQL interface on top of Flink that was started by Stephan. > >>>>> > >>>>> The document proposes an architecture that is centered around > >>>>> Apache Calcite. Calcite is an Apache top-level project and > >>>>> includes a SQL > >>>> parser, > >>>>> a semantic validator for relational queries, and a rule- and > >> cost-based > >>>>> relational optimizer. Calcite is used by Apache Hive and Apache > >>>>> Drill (among other projects). In a nutshell, the plan is to > >>>>> translate Table > >>> API > >>>>> and SQL queries into Calcite's relational expression trees, > >>>>> optimize > >>>> these > >>>>> trees, and translate them into DataSet and DataStream programs.The > >>>> document > >>>>> breaks down the work into several tasks and subtasks. > >>>>> > >>>>> Please review the design document and comment. > >>>>> > >>>>> -- > > >>>>> > >>>> > >>> > >> https://docs.google.com/document/d/1TLayJNOTBle_-m1rQfgA6Ouj1oYsfqRjP > >> cp1h2TVqdI/edit?usp=sharing > >>>>> > >>>>> Unless there are major concerns with the design, Timo and I want > >>>>> to > >>> start > >>>>> next week to move the current Table API on top of Apache Calcite > >> (Task > >>> 1 > >>>> in > >>>>> the document). The goal of this task is to have the same > >> functionality > >>> as > >>>>> currently, but with Calcite in the translation process. This is a > >>>> blocking > >>>>> task that we hope to complete soon. Afterwards, we can > >>>>> independently > >>> work > >>>>> on different aspects such as extending the Table API, adding a SQL > >>>>> interface (basically just a parser), integration with external > >>>>> data sources, better code generation, optimization rules, > >>>>> streaming > >> support > >>>> for > >>>>> the Table API, StreamSQL, etc.. > >>>>> > >>>>> Timo and I plan to work on a WIP branch to implement Task 1 and > >>>>> merge > >>> it > >>>> to > >>>>> the master branch once the task is completed. Of course, everybody > >>>>> is welcome to contribute to this effort. Please let us know such > >>>>> that we > >>> can > >>>>> coordinate our efforts. > >>>>> > >>>>> Thanks, > >>>>> Fabian > > Regards, > Chiwan Park > > >
