Hi Martijn, Thanks for your feedback to this FLIP. So far as I know, MATCH_RECOGNIZE does not support multiple & dynamic pattern matching, and there is no other standard SQL grammar that supports this function. In this case I'm not sure whether we would like Table/SQL API to support this newly introduced feature, as it means the API will do more than SQL the language has defined. If it is true, then we need to introduce a new SQL grammar on our own.
As for implementation, I think it is possible to link the operators created in our DataStream implementation with the newly introduced SQL grammar. Input parameters required by DataStream implementation. like the discoverer factory, can be passed in through the SQL grammar. Apart from that, I believe most implementation detail of this function in SQL is identical to that of DataStream. Generally speaking, when it comes to Table/SQL implementation I believe what we need to do is to introduce a new SQL grammar and link the grammar to internal DataStream implementation. I have summarized these thoughts in the FLIP's Future Work part and you can have a look. Best regards, Yunfeng On Fri, Dec 10, 2021 at 10:58 PM Martijn Visser <mart...@ververica.com> wrote: > Hi Timo, > > My expectation is that we should at least think of what and how it should > work for SQL/Table API, especially since that's our recommendation for new > users.That doesn't mean that everything needs to be implemented at the same > time. > > My worry is that in order for a SQL / Table implementation to work, you > would need to introduce changes that could be avoided upfront. > > Best regards, > > Martijn > > Op vr 10 dec. 2021 om 14:38 schreef Timo Walther <twal...@apache.org> > > > Hi all, > > > > I don't think it is easy to integrate this kind of functionality in > > SQL/Table API. The CEP library can be more powerful than > > MATCH_RECOGNIZE. I haven't taken a look at the FLIP yet. But I would be > > fine to leave Table API/SQL up for future work and a separate FLIP. > > > > Regards, > > Timo > > > > > > On 10.12.21 13:44, Ingo Bürk wrote: > > > Hi, > > > > > > I agree with Martijn on this. The lack of parity across major APIs is a > > > frequent cause of user questions and friction, forcing users to switch > > > between APIs etc. I would therefore also suggest expanding the scope to > > > cover Table API + SQL as well. In general we should probably split > FLIPs > > > on a functional level and not between APIs. > > > > > > > > > Best > > > > > > Ingo > > > > > > On 10.12.21 13:24, Martijn Visser wrote: > > >> Apologies, I do see SQL mentioned at the bottom for new rules, but I > > >> don't > > >> think it's a good idea to have these changes only for the DataStream > > >> API in > > >> the beginning. This would increase sparsity in Flink, which we should > > >> avoid. > > >> > > >> Best regards, > > >> > > >> Martijn > > >> > > >> On Fri, 10 Dec 2021 at 13:19, Martijn Visser <mart...@ververica.com> > > >> wrote: > > >> > > >>> Hi Yunfeng, > > >>> > > >>> Thanks for creating the FLIP. I don't see any mention of SQL's > > >>> MATCH_RECOGNIZE implementation in the FLIP and I think that any > > >>> change in > > >>> CEP should be available to both DataStream and SQL/Table API users. > > >>> Can you > > >>> elaborate on that? > > >>> > > >>> Best regards, > > >>> > > >>> Martijn > > >>> > > >>> On Fri, 10 Dec 2021 at 12:16, Yunfeng Zhou < > > flink.zhouyunf...@gmail.com> > > >>> wrote: > > >>> > > >>>> Hi all， > > >>>> > > >>>> I'm opening this thread to propose the design to support multiple > > >>>> rule & > > >>>> dynamic rule changing in the Flink-CEP project, as described in > > >>>> FLIP-200 > > >>>> [1] > > >>>> . > > >>>> > > >>>> Currently Flink CEP only supports having a single pattern inside a > > >>>> CepOperator and does not support changing the pattern dynamically. > In > > >>>> order > > >>>> to reduce resource consumption and to experience shorter downtime > > >>>> during > > >>>> pattern updates, there is a growing need in the production > environment > > >>>> that > > >>>> expects CEP to support having multiple patterns in one operator and > to > > >>>> support dynamically changing them. Therefore I propose to add > certain > > >>>> infrastructure as described in FLIP-200 to support these > > >>>> functionalities. > > >>>> > > >>>> Please feel free to reply to this email thread. Looking forward to > > your > > >>>> feedback! > > >>>> > > >>>> [1] > > >>>> > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=195730308 > > >>>> > > >>>> > > >>>> Best regards, > > >>>> > > >>>> Yunfeng > > >>>> > > > > > > > -- > > Martijn Visser | Product Manager > > mart...@ververica.com > > <https://www.ververica.com/> > > > Follow us @VervericaData > > -- > > Join Flink Forward <https://flink-forward.org/> - The Apache Flink > Conference > > Stream Processing | Event Driven | Real Time >
