Kafka + Flink, Basic Questions

2016-10-30 Thread Dromit
Hello, I'm writing an application to generate reports of a real time stream of sale records (product, quantity, price, time) that I get from a websocket, and I plan to use Kafka and Flink to do so. The result of the pipeline should be, for instance, "the average price for a certain product in the

Re: Retrieving a single element from a DataSet

2016-10-30 Thread Greg Hogan
I created FLINK-4965 "AllPairsShortestPaths" and FLINK-4966 "BetweennessCentrality". On Wed, Oct 26, 2016 at 4:39 PM, Greg Hogan wrote: > It sounds like you want to use an all-pairs shortest paths algorithm. This > would be a great contribution to Gelly! > https://en.wikipedia.org/wiki/Shortes

Re: Looping over a DataSet and accesing another DataSet

2016-10-30 Thread Gábor Gévay
Hello, In Flink, one often used way to access data from multiple DataSets at the same time is to perform a join (Flink actually calls equi-joins [1] just "join"), just as in the database world. For example, in the algorithm that you linked, you access A[u] for every edge (u,v). I assume that you

Looping over a DataSet and accesing another DataSet

2016-10-30 Thread otherwise777
Currently i'm trying to implement this algorithm [1] which requires me to loop over one DataSet (the edges) and access another DataSet (the vertices), for this loop i use a Mapping (i'm not sure if this is the correct way of looping over a DataSet) but i don't know how to access the elements of ano