Hi Sameer,
I just replied to the earlier post, but I will copy it here:
We also have the same requirement - we want to allow the user to change the
matching patterns and have them take effect immediately. I'm wondering whether
the proposed trigger DSL takes us one step closer:(I don't think it solves the
problem) or we have to dynamically generate Flink job JAR files when the
matching rules/patterns are changed and submit them to Flink.
I had thought about using a similar approach, but it is quite restrictive
because you cannot use the power for Flink CEP engine with this approach. For
example, I want to be able to use followedBy, next, notFollowedBy (in future)
operators to detect the patterns and these matching patterns need to be
user-cofigurable/dynamic/hot deployable. The simple attribute-based
rules/patterns that you specified can be made dynamic as you mentioned but the
rules/patterns that use not just the current event attributes, but also past
events (e.g. followedBy) are much harder to make them dynamic without some help
from Flink that implements the CEP operators.
- LF
From: Sameer W <sam...@axiomine.com>
To: "user@flink.apache.org" <user@flink.apache.org>
Sent: Tuesday, October 11, 2016 2:23 PM
Subject: Re: What is the best way to load/add patterns dynamically (at
runtime) with Flink?
I have used a JavaScript engine in my CEP to evaluate my patterns. Each event
is a list of named attributes (HashMap like). And event is attached to a list
of rules expressed as JavaScript code (See example below with one rule but I
can match as many rules). The rules are distributed over a connected stream
which allow it to change over time. This is how I do it to keep my rules
dynamic. If someone has a better way I would love to hear it as well.
private transient ScriptEngineManager factory = new ScriptEngineManager();
private transient ScriptEngine engine = factory.getEngineByName("JavaScript");
/*Inside open*/
factory = new ScriptEngineManager();
/*Close open*/
/*Inside my operator*/
engine = factory.getEngineByName("JavaScript");
engine.put("evt", value.f1); //value.f1 contains a JSON version of my HashMap
of attributes
engine.eval(value.f2.rule); //f2.last contains the rule which is evaluated by
the JavaScript Engine/*Sample JavaScript contained in the call -
engine.eval(value.f2.rule); is shown below (not the "evt" variable in the
JavaScript and the the preceding line - engine.put("evt", value.f1);
var evt=JSON.parse(evt);var result = evt.temperature>50 && evt.pressure<900
*/
boolean ret = (boolean)engine.get("result");
if(ret) /*Rule is Matched*/
> On Oct 11, 2016, at 5:01 PM, PedroMrChaves <pedro.mr.cha...@gmail.com> wrote:
>
> Hello,
>
> I am new to Apache Flink and am trying to build a CEP using Flink's API. One
> of the requirements is the ability to add/change patterns at runtime for
> anomaly detection (maintaining the systems availability). Any Ideas of how
> could I do that?
>
> For instance, If I have a stream of security events (accesses,
> authentications ,etc.) and a pattern for detecting anomalies I would like to
> be able to change that pattern parameters, for instance instead of detecting
> the occurrence of events A->B->C I would like to change the condition on B
> to B’ in order to have a new rule. Moreover, I would like to be able to
> create new patterns dynamically as new use cases arise.
>
> Best Regards,
> Pedro Chaves
>
>
>
>
> --
> View this message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/What-is-the-best-way-to-load-add-patterns-dynamically-at-runtime-with-Flink-tp9461.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at
> Nabble.com.
