Hey,
Wow, Vasia, a GC limit error in the Optimizer (pre-flight), not the runtime.
>
> That is cannot have been easy to produce ;-) The program does not really
> look that tricky, I need to take some closer look at this...
>
It was pretty easy actually :P
The program is really simple: in each ite
Wow, Vasia, a GC limit error in the Optimizer (pre-flight), not the runtime.
That is cannot have been easy to produce ;-) The program does not really
look that tricky, I need to take some closer look at this...
BTW: Before this algorithm runs well, we really need pinning intermediate
results in m
Hi,
I'm back looking into this and I tried out the for-loop approach that was
suggested above.
I implemented a simple algorithm, k-core, which computes the k-core of a
graph by iteratively filtering out vertices with degree less than k.
You can find the code in [1].
Unfortunately, this is giving
I see, thanks a lot for the answers!
To rephrase my original question, would it make sense to define a
closed-loop iteration where the state is the whole graph?
If you want to take a look at the current implementation of DMST using
delta iteration, Andra has made a PR [1].
On a high-level, this al
I see that's cool :-)
So, what is the advantage of closed-loop versus for-loop iterations?
Custom convergence criteria / aggregators and more efficient execution
plans?
On 23 February 2015 at 15:01, Stephan Ewen wrote:
> For loops are basically rolled out - they yield long execution plans.
>
> O
Closed-loop iterations are much more efficient right now. Long for loops
suffer from memory fragmentation (an issue that is in the list to fix).
Also, closed loops can be stateful (delta iterations) and do not require
task re-deployment.
On Mon, Feb 23, 2015 at 4:15 PM, Vasiliki Kalavri wrote:
For loops are basically rolled out - they yield long execution plans.
On Mon, Feb 23, 2015 at 2:44 PM, Vasiliki Kalavri wrote:
> for-loop iterations could cover some cases, I guess, when the number of
> iterations is known beforehand.
> Are there currently any restrictions on what can be used in
for-loop iterations could cover some cases, I guess, when the number of
iterations is known beforehand.
Are there currently any restrictions on what can be used inside a for-loop?
How are they translated into execution plans?
On 23 February 2015 at 13:08, Stephan Ewen wrote:
> Some things may no
Some things may not work well as "closed-loop" iterations.
Is it possible to express those as for-loop iterations?
On Mon, Feb 23, 2015 at 1:03 PM, Vasiliki Kalavri wrote:
> Hi Stephan,
>
> yes, this would work for the cases where an algorithm only updates the
> vertex values or only updates th
Hi Stephan,
yes, this would work for the cases where an algorithm only updates the
vertex values or only updates the edge values.
What we would like to also support is
(a) algorithms where both vertices and edges are updated in one iteration
(b) algorithms where the graph structure changes from o
As a workaround, it should always work to get the Edge and Vertex data set
from the graph and use the regular Fink iteration operators?
On Sun, Feb 22, 2015 at 4:53 PM, Vasiliki Kalavri wrote:
> Hi,
>
> yes, I was referring to the parallel Boruvka algorithm. There are several
> ways to implemen
Hi,
yes, I was referring to the parallel Boruvka algorithm. There are several
ways to implement this one in Flink and I believe that the one described in
the paper (vertex-centric) is not the most elegant one :)
Andra is now working on an idea that uses the delta iteration abstraction
and we beli
Hi Alex,
Vasia is talking about the second version(presented Friday) of Parallel
Boruvka, which can be found here:
https://github.com/TU-Berlin-DIMA/IMPRO-3.WS14/pull/59
I will propose the third, non-Pregel like approach directly to Gelly soon.
If you have additional questions, I will be happy t
Hi Vasia,
I am trying to look at the problem in more detail. Which version of the MST
are you talking about?
Right now in the Gelly repository I can only find the SSSP example
(parallel Bellman-Ford) from Section 4.2 in [1].
However, it seems that the issues encountered by Andra are related to t
Hello beautiful Flink people,
during the past few days, Andra and I have been discussing about how to
extend Gelly's iteration methods.
Alexander's course (and his awesome students) has made it obvious that
vertex-centric iterations are not the best fit for algorithms which don't
follow the commo
15 matches
Mail list logo
