Hi,
I've been using drools in a few projects. Embedding Drools in Flink is
certainly possible, however there are a few points you have to consider.
- are you going to use Drools in stateful or stateless way? The blog
post you mention uses flink in stateless way - that is, each event is
processed separately and
each drools session behaves like a bunch of if's. This is
straightforward, it also shouldn't be problematic performance wise
OTOH, if you want to use stateful sessions things are getting more
complicated - because if you want to play well with Flink you'd have to
integrate drools sessions with checkpoints in Flink - and that can be a
bit more tricky (although certainly possible) - especially when it comes
to size of session, partitioning and so on.
Even more difficult would be to use Drools CEP features - because then
you'd have to consider how to handle time (e.g. during restore??).
To be honest, I wouldn't try to integrate Drools CEP with Flink - Flink
has it's own time handling, own CEP engine and mixing those two can lead
to tricky issues...
Even if you decide to use stateless sessions there's issue with
deployment. If you don't want to redeploy Flink process each time one of
the users change rules, you'd have to implement some notion of detecting
changes (and storing rules somewhere?) and recompiling rules on each
task manager. Again - this is certainly doable and possible, but
requires some thinking about the design.
To sum up - there are many ways you can integrate drools with flink. I'd
start with something simple - exactly as Robert wrote
hope this helps a bit and good luck,
maciek
On Tue, Jun 21, 2016 at 7:55 AM, Anton <kurren...@gmail.com> wrote:
Hello
Firstly, I am an absolute Flink newbie.
I am interested in using Drools in Flink - in a similar case to what is
described in this blog, where Drools is used in Spark.
http://blog.cloudera.com/blog/2015/11/how-to-build-a-complex-event-processing-app-on-apache-spark-and-drools/
The basic idea is that Drools can be used to reason over streaming data.
The high-level use case is, there are several hundred users who want to
write rules to be notified on events related to changes in specific
streams. For example, notify me when a specific stock price changes by so
much.
Due to the number of users, the more end-user focused syntax of Drools, and
the number of rule changes, doing this in Drools makes more sense than to
write and deploy plain-old-flink (apologies, am not familiar with the
correct term for a flink process).
Also, as Drools has some powerful CEP operators, it could be very
interested to have these available in Flink too.
My question, therefore, is, very general - how best to integrate Drools
into Flink? Where should I start?
Thanks and regards
Anton
