Re: Complex batch workflow needs (too) much time to create executionPlan

2016-09-26 Thread Markus Nentwig
Hi Fabian, at first, sorry for the late answer. The given execution plan was created after 20 minutes, only one vertex centric iteration is missing. I can optimize the program because some operators are only needed to create intermediate debug results, still, it's not enough to run as one Flink j

Complex batch workflow needs (too) much time to create executionPlan

2016-08-22 Thread Markus Nentwig
Hello Flink community, I created a slightly long batch workflow for my use case of clustering vertices using Flink and Gelly. Executing each of the workflow parts individually (and write intermediate results to disk) works as suspected. When combining workflow parts to longer jobs, I noticed that