Hi All, One way to try to think about it with StateFun, is to represent the Graph vertices as stateful functions instances. Unlike other frameworks an instance of a function does not take any resources while idle, and potentially you can have many millions of those. A state for each vertex might be a list of adjacent vertices, and potentially a timer so that they won’t linger for to long. You would still have to think of what kind of graph algorithm to apply here.
I hope it helps, Igal. On Thursday, April 9, 2020, Robert Metzger <rmetz...@apache.org> wrote: > Hey Max, > > 1) Stateful functions has been released now: https://mvnrepository. > com/artifact/org.apache.flink/statefun-flink-core > See also: https://flink.apache.org/news/2020/04/07/release- > statefun-2.0.0.html > Getting Started: https://ci.apache.org/projects/flink/flink- > statefun-docs-release-2.0/getting-started/java_walkthrough.html > > Please give us honest feedback on the "onboarding experience" with > stateful functions. We are very eager to make the experience as smooth as > possible :) > > 2) What do you consider large state? The Flink runtime itself can handle > large events / messages (1GB should work). I'm not sure about statefun, but > I will try to get an answer for you :) > > Best, > Robert > > > On Tue, Apr 7, 2020 at 9:31 AM m@xi <makisnt...@gmail.com> wrote: > >> Hello Robert >> >> Thanks to your reply I discovered the Stateful Functions which I believe >> is >> a quite powerful tool. I have some questions: >> >> 1) As you said, "the community is currently in the process of releasing >> the >> first Apache release of StateFun and it should hopefully be out by the end >> of this week". Does this mean that it will become available in Maven >> Repository? >> >> Because I can't find it while searching in >> https://mvnrepository.com/artifact/org.apache.flink?sort=newest >> or >> use the API in my intellij project when I import the dependencies in my >> POM >> file. >> >> I though of dowloading the code from >> https://ci.apache.org/projects/flink/flink-statefun-docs-master/, >> compiling >> it with *mvn clean package* and then import the produced jar file to my >> intellij project as an External Library. Is this what you might recommend >> for now? >> >> 2) I came across this tutorial by Stefan on stateful functions >> https://www.youtube.com/watch?v=fCeHCMJXXM0 where he mentions that >> arbitrary >> communication between nodes/functions/actors is essentially made possible >> by >> introducing feedback loops to the DAG Flink topology (well now it has >> circles I guess :P) to simulate the arbitrary messasing defined in the >> StateFun topology. >> >> So the message passing is done by "feedback and tuple rerouting" and not >> with MPI. Do you think (or have tested) if one may *efficiently* >> send/receive (potentially large) state, like graph state which is the use >> case of this post? Or it is more suitable for sending control messages >> between actors/functions? >> >> Thanks a lot in advance. >> >> Best, >> Max >> >> >> >> -- >> Sent from: http://apache-flink-user-mailing-list-archive.2336050. >> n4.nabble.com/ >> >
