Re: Flink DataSet Iterate updating additional variable

2020-07-14 Thread Khachatryan Roman
Hi Antonio, Yes, you are right. Revisiting your question, I'm wondering whether it's possible to partition speeds and nodes in the same way (stably across iterations)? (I'm assuming a distributed setup) If not, each iteration would have to wait for *all* subtasks of the previous iteration to finis

Re: Flink DataSet Iterate updating additional variable

2020-07-14 Thread Antonio Martínez Carratalá
Hi Roman, Thank you for your quick reply, but as far as I know broadcast variables cannot be written, my problem is that I need to update the value of the speed variable to use it in the next iteration. Iterate only has one input dataset and propagates it to the next iteration using closeWith(),

Re: Flink DataSet Iterate updating additional variable

2020-07-13 Thread Khachatryan Roman
Hi Antonio, Please take a look at broadcast variables: https://ci.apache.org/projects/flink/flink-docs-stable/dev/batch/#broadcast-variables Regards, Roman On Mon, Jul 13, 2020 at 3:49 PM Antonio Martínez Carratalá < amarti...@alto-analytics.com> wrote: > Hello > > I'm trying to implement the

Flink DataSet Iterate updating additional variable

2020-07-13 Thread Antonio Martínez Carratalá
Hello I'm trying to implement the ForceAtlas2 (graph layout) algorithm in Flink using datasets, it is an iterative algorithm and I have most of it ready, but there is something I don't know how to do. Apart from the dataset with the coordinates (x,y) of each node I need an additional variable to r