Re: problem with increase job parallelism

2017-10-20 Thread Fabian Hueske
Hi Lei, setting explicit operator ID should solve this issue. As far as I know, the auto-generated operator id also depended on the operator parallelism in previous versions of Flink (not sure until which point). Which version are you running? Best, Fabian 2017-10-17 3:15 GMT+02:00 Lei Chen :

problem with increase job parallelism

2017-10-16 Thread Lei Chen
Hi, We're trying to implement some module to help autoscale our pipeline which is built with Flink on YARN. According to the document, the suggested procedure seems to be: 1. cancel job with savepoint 2. start new job with increased YARN TM number and parallelism. However, step 2 always gave er