You should be able to use a KeyedProcessFunction
<https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/operators/process_function.html#the-processfunction>for
that.
Find matching elements via keyBy() on the first field.
Aggregate into ValueState, send alert if necessary.
Upon encountering a new key, setup a timer to remove the entry in 24h.
On 08.02.2019 07:43, Titus Rakkesh wrote:
Dears,
I have a data stream continuously coming,
DataStream<Tuple3<String, Integer, Double>> splitZTuple;
Eg - (775168263,113182,0.0)
I have to store this for 24 hrs expiry in somewhere (Window or
somewhere) to check against another stream.
The second stream is
DataStream<Tuple2<String, Double>> splittedVomsTuple which also
continuously receiving one.
Eg. (775168263,100.0)
We need to accumulate the third element in (775168263,113182,*/0.0/*)
in the WINDOW (If the corresponding first element match happened with
the incoming second streams second element 775168263,*/100.0/*)
While keeping this WINDOW session if any (775168263,113182,*/175/*)
third element in the Window Stream exceed a value (Eg >150) we need to
call back a another rest point to send an alert -------
(775168263,113182,*/175/*) match the criteria. Simply a CEP call back.
In Flink how we can do this kind of operations? Or do I need to think
about any other framework? Please advise.
Thanks...
