Hi, Here's our use case : We are planning to build a rule based engine on top of flink with huge number of rules(1000s). the rules could be stateless or stateful. Example stateless rule is : A.id = 3 && A.name = 'abc' || A.color = red. Example stateful rule is : A is event.id =3, B is event.name = 'abc', C is event.color = red and we are looking for pattern AB*C over time window of 1 hour.
Now we have tried to use flink CEP for this purpose and program crashed because of lot of threads. The explanation is : every application of CEP.pattern creates a new operator in the graph and flink can't support that many vertices in a graph. Other approach could be to use processFunction in flink, but still to run the rules on events stream you'd have to use some sort of CEP or write your own. My question is, does anybody have any other suggestions on how to achieve this ? Any other CEPs that integrate and work better with flink (siddhi, jasper, drools) ? Any experience would be helpful.
