Hello squirrels, I have been discussing with the Apache Tinkerpop [1] community regarding an integration with Flink/Gelly. You can read our discussion in [2].
Tinkerpop has a graph traversal machine called Gremlin, which supports many high-level graph processing languages and runs on top of different systems (e.g. Giraph, Spark, Graph DBs). You can read more in this great blog post [3]. The idea is to provide a FlinkGraphComputer implementation, which will add Gremlin support to Flink. I believe Tinkerpop is a great project and I would love to see an integration with Gelly. Before we move forward, I would like your input! To me, it seems that this addition would nicely fit in flink-contrib, where we also have connectors to other projects. If you agree, I will go ahead and open a JIRA about it. Thank you! -Vasia. [1]: https://tinkerpop.incubator.apache.org/ [2]: https://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/201511.mbox/%3ccanva_a390l7g169r8sn+ej1-yfkbudlnd4td6atwnp0uza-...@mail.gmail.com%3E [3]: http://www.datastax.com/dev/blog/the-benefits-of-the-gremlin-graph-traversal-machine On 25 November 2015 at 16:54, Vasiliki Kalavri <vasilikikala...@gmail.com> wrote: > Hi James, > > I've just subscribed to the Tinkerpop dev mailing list. Could you please > send a reply to the thread, so then I can reply to it? > I'm not sure how I can reply to the thread otherwise... > I also saw that there is a grafos.ml project thread. I could also provide > some input there :) > > Thanks! > -Vasia. > > On 25 November 2015 at 15:09, James Thornton <james.thorn...@gmail.com> > wrote: > >> Hi Vasia - >> >> Yes, a FlinkGraphComputer should be a straight-forward first step. Also, >> on >> the Apache Tinkerpop dev mailing list, Marko thought it might be cool if >> there was a "Graph API" similar to the "Table API" -- hooking in Gremlin >> to >> Flink's fluent API would give Flink users a full graph query language. >> >> Stephen Mallette is a TinkerPop core contributor, and he has already >> started working on a FlinkGraphComputer. There is a Flink/Tinkerpop thread >> on the TinkerPop dev list -- it would be great to have you part of the >> conversation there too as we work on the integration: >> >> http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/ >> >> Thanks, Vasia. >> >> - James >> >> >> On Mon, Nov 23, 2015 at 10:28 AM, Vasiliki Kalavri < >> vasilikikala...@gmail.com> wrote: >> >> > Hi James, >> > >> > thank you for your e-mail and your interest in Flink :) >> > >> > I've recently taken a _quick_ look into Apache TinkerPop and I think >> it'd >> > be very interesting to integrate with Flink/Gelly. >> > Are you thinking about something like a Flink GraphComputer, similar to >> > Giraph and Spark GraphComputer's? >> > I believe such an integration should be straight-forward to implement. >> You >> > can start by looking into Flink iteration operators [1] and Gelly >> iteration >> > abstractions [2]. >> > >> > Regarding Apache Geode, I'm not familiar with project, but I'll try to >> take >> > a look in the following days! >> > >> > Cheers, >> > -Vasia. >> > >> > >> > [1]: >> > >> > >> https://ci.apache.org/projects/flink/flink-docs-master/apis/programming_guide.html#iteration-operators >> > [2]: >> > >> > >> https://ci.apache.org/projects/flink/flink-docs-master/libs/gelly_guide.html#iterative-graph-processing >> > >> > >> > On 20 November 2015 at 08:32, James Thornton <james.thorn...@gmail.com> >> > wrote: >> > >> > > Hi - >> > > >> > > This is James Thornton (espeed) from the Apache Tinkerpop project ( >> > > http://tinkerpop.incubator.apache.org/). >> > > >> > > The Flink iterators should pair well with Gremlin's Graph Traversal >> > Machine >> > > ( >> > > >> > > >> > >> http://www.datastax.com/dev/blog/the-benefits-of-the-gremlin-graph-traversal-machine >> > > ) >> > > -- it would be good to coordinate on creating an integration. >> > > >> > > Also, Apache Geode made a splash today on HN ( >> > > https://news.ycombinator.com/item?id=10596859) -- connecting Geode >> and >> > > Flink would be killer. Here's the Geode/Spark connector for >> refefference: >> > > >> > > >> > > >> > > >> > >> https://github.com/apache/incubator-geode/tree/develop/gemfire-spark-connector >> > > >> > > - James >> > > >> > >> > >
