Hi Jakub, I had a look at the changes of Flink 1.5 [1] and didn't find anything obvious. Something that might cause a different behavior is the new deployment and process model (FLIP-6).
In Flink 1.5, there is a switch to disable it and use the previous deployment mechanism. You could try to disable the new new model [2] and see if this cause the performance issue. Note that the legacy mode was removed in one of the later versions. Best, Fabian [1] https://flink.apache.org/news/2018/05/25/release-1.5.0.html [2] https://ci.apache.org/projects/flink/flink-docs-release-1.5/release-notes/flink-1.5.html#update-configuration-for-reworked-job-deployment Am Do., 24. Okt. 2019 um 19:37 Uhr schrieb Jakub Danilewicz < jdanilew...@alto-analytics.com>: > Hi, > > I have recently tried to upgrade Flink from 1.2.0 to the newest version > and noticed that starting from the version 1.5 the performance is much > worse when processing fixed graphs in a standalone JVM environment (Java > 8). > > This affects all the use-cases when a Gelly graph (pre-built from a fixed > collection of nodes/edges) gets processed by any of our custom algorithms > (VertexCentric, ScatterGather or GSA), especially when using parallel > processing for a local ExecutionEnvironment. The processing times > (compared to the versions <= 1.4.2) double/triple, while CPU and memory > consumption increase significantly. > > Are there any fine-tuning steps/tricks for the job processing engine > behind Flink 1.5+ that would improve the performance in the scenarios > described above? > > Best, > > Jakub >
