Re: Retrieving a single element from a DataSet

2016-11-10 Thread otherwise777
Hey there, I don't really understand what Broadcast does, does it in a way export the elements from a DataSet in a Collection? Because then it might be what i'm looking for. when implementing algorithms in Flink Gelly i keep getting stuck on what i cannot do with DataSets. For example, if i want

Re: Retrieving a single element from a DataSet

2016-11-05 Thread Vasiliki Kalavri
Hi all, @Wouter: I'm not sure I completely understand what you want to do, but would broadcast variables [1] help? @all: All-pairs-shortest-paths and betweenness centrality are very challenging algorithms to implement efficiently in a distributed way. APSP requires each vertex to store distances

Re: Retrieving a single element from a DataSet

2016-11-04 Thread Greg Hogan
The tickets are in Flink's Jira: https://issues.apache.org/jira/browse/FLINK-4965 https://issues.apache.org/jira/browse/FLINK-4966 Are you looking to process temporal graphs with the DataStream API? On Fri, Nov 4, 2016 at 5:52 AM, otherwise777 wrote: > Cool, thnx for that, > > I tried searc

Re: Retrieving a single element from a DataSet

2016-11-04 Thread otherwise777
Cool, thnx for that, I tried searching for it in teh github but couldn't find it, do you have the url by any chance? I'm going to try to implement such an algorithm for temporal graphs -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Retrievi

Re: Retrieving a single element from a DataSet

2016-10-30 Thread Greg Hogan
I created FLINK-4965 "AllPairsShortestPaths" and FLINK-4966 "BetweennessCentrality". On Wed, Oct 26, 2016 at 4:39 PM, Greg Hogan wrote: > It sounds like you want to use an all-pairs shortest paths algorithm. This > would be a great contribution to Gelly! > https://en.wikipedia.org/wiki/Shortes

Re: Retrieving a single element from a DataSet

2016-10-26 Thread Greg Hogan
It sounds like you want to use an all-pairs shortest paths algorithm. This would be a great contribution to Gelly! https://en.wikipedia.org/wiki/Shortest_path_problem#All-pairs_shortest_paths On Wed, Oct 26, 2016 at 9:29 AM, otherwise777 wrote: > That is indeed not the nice way to do it because

Re: Retrieving a single element from a DataSet

2016-10-26 Thread otherwise777
That is indeed not the nice way to do it because it will create an executionplan just to get that value, but it does work, so thnx for that A more concrete example for what i want, In gelly you have the SingleSourceShortestPaths algorith which requires the sourceVertexId, now i want to execute a m

Re: Retrieving a single element from a DataSet

2016-10-26 Thread Sebastian Neef
Hi, I'm also interested in that question/solution. For now, my workaround looks like this: > DataSet<...> .filter(... object.Id == NeededElement.Id ... ).collect().get(0) I filter the DataSet for the element I want to find, collect it into a List which then returns the first element. That's a