Re: [Gelly]Distributed Minimum Spanning Tree Example

2015-02-15 Thread Stephan Ewen
I think Fabian has a good direction. @Andra, are you using a mapPartition() operation? If yes, it is really non-deterministic, unless you explicitly call a partitioning before (rebalancing being a special case thereof). If you are not using that, can you point us to the code and give us a bit mor

Re: [Gelly]Distributed Minimum Spanning Tree Example

2015-02-14 Thread Fabian Hueske
Hi Andra, I haven't had a detailed look at Gelly and its functions, but Flink has only few operators which can cause undeterministic behavior. In general, user code should be implemented without side effects, i.e., the result of each function call may only depend on its arguments. This principle g

[Gelly]Distributed Minimum Spanning Tree Example

2015-02-14 Thread Andra Lungu
Hey guys, As I previously said, I have had some problems getting this DMST algorithm to be fully functional, either with Flink 0.8 or with Flink 0.9. My latest problem(and I have been debugging this for quite some days) was that for the test I wrote that extended MultipleProgramsTestBase, the Exe