Re: Complex graph-based sessionization (potential use for stateful functions)

2020-04-26 Thread m@xi
Hi Igal, Thanks a lot for your answer. I believe you are one of the core developers behind the interesting statefun. Your suggestion is really nice and as you say, one way is to tailor the graph processing to the philosophy of SF. Though, if one vertex is a stateful function, then heavy hitter n

Re: Fwd: Complex graph-based sessionization (potential use for stateful functions)

2020-04-26 Thread m@xi
Hi Robert, Thanks a lot for your reply. 1) Now statefun packages are in the MVN repository, so probably they needed some time to really be included there after your official release. 2) Alinged to the topic of the thread, I am referring to state of massive graph streams. To enable distribut

Re: Complex graph-based sessionization (potential use for stateful functions)

2020-04-09 Thread Igal Shilman
Hi All, One way to try to think about it with StateFun, is to represent the Graph vertices as stateful functions instances. Unlike other frameworks an instance of a function does not take any resources while idle, and potentially you can have many millions of those. A state for each vertex might be

Re: Fwd: Complex graph-based sessionization (potential use for stateful functions)

2020-04-09 Thread Robert Metzger
Hey Max, 1) Stateful functions has been released now: https://mvnrepository.com/artifact/org.apache.flink/statefun-flink-core See also: https://flink.apache.org/news/2020/04/07/release-statefun-2.0.0.html Getting Started: https://ci.apache.org/projects/flink/flink-statefun-docs-release-2.0/getting

Re: Fwd: Complex graph-based sessionization (potential use for stateful functions)

2020-04-07 Thread m@xi
Hello Robert Thanks to your reply I discovered the Stateful Functions which I believe is a quite powerful tool. I have some questions: 1) As you said, "the community is currently in the process of releasing the first Apache release of StateFun and it should hopefully be out by the end of this wee

Fwd: Complex graph-based sessionization (potential use for stateful functions)

2020-03-31 Thread Robert Metzger
Forwarding Seth's answer to the list -- Forwarded message - From: Seth Wiesman Date: Tue, Mar 31, 2020 at 4:47 PM Subject: Re: Complex graph-based sessionization (potential use for stateful functions) To: Krzysztof Zarzycki Cc: user , Hi Krzysztof, This is a great use

Complex graph-based sessionization (potential use for stateful functions)

2020-03-30 Thread Krzysztof Zarzycki
Hi! Interesting problem to solve ahead :) I need to implement a streaming sessionization algorithm (split stream of events into groups of correlated events). It's pretty non-standard as we DON'T have a key like user id which separates the stream into substreams which we just need to chunk based on