Thanks Till! To execute the above (using Kubernetes), one would enter the running JobManager service and execute it? The following REST API call does the same */jobs/:jobid/rescaling*?
I assume it changes the base parallelism, but what it will do if I had already set the parallelism of my operators? e.g. .source(..) .setParallelism(3) .setUID(..) .map(..) .setParallelism(8) .setUID(..) .sink(..) .setParallelism(3) .setUID(..) I think it would be a good idea to have */jobs/:jobid/rescaling,* additionally requiring the *operatorUID* as a queryParameter*, *so that the parallelism of specific operators could be changed. Best, Alex. On Tue, 15 Jan 2019 at 10:27, Till Rohrmann <trohrm...@apache.org> wrote: > Hi Alexandru, > > you can use the `modify` command `bin/flink modify <JOB_ID> --parallelism > <PARALLELISM>` to modify the parallelism of a job. At the moment, it is > implemented as first taking a savepoint, stopping the job and then > redeploying the job with the changed parallelism and resuming from the > savepoint. > > Cheers, > Till > > On Mon, Jan 14, 2019 at 4:21 PM Dawid Wysakowicz <dwysakow...@apache.org> > wrote: > >> Hi Alexandru >> >> As for 2, generally speaking the number of required slots depends on >> number of slot sharing groups. By default all operators belong to the >> default slot sharing group, that means a job requires as many slots as >> maximal parallelism in the job. More on the distributed runtime you can >> read here[1] >> >> As for 1 I cc'ed Gary and Till who might better answer your question. >> >> [1] >> https://ci.apache.org/projects/flink/flink-docs-release-1.7/concepts/runtime.html#task-slots-and-resources >> >> Best, >> >> Dawid >> On 14/01/2019 15:26, Alexandru Gutan wrote: >> >> Hi everyone! >> >> 1. Is there a way to increase the parallelism (e.g. through REST) of some >> operators in a job without re-deploying the job? I found this >> <https://stackoverflow.com/questions/50719147/apache-flink-guideliness-for-setting-parallelism> >> answer which mentions scaling at runtime on Yarn/Mesos. Is it possible? >> How? Support for Kubernetes? >> 2. What happens when the number of parallel operator instances exceeds >> the number of task slots? For example: a job with a source (parallelism 3), >> a map (parallelism 8), a sink (parallelism 3), total of *14* operator >> instances and a setup with *8* task slots. Will the operators get >> chained? What if I disable operator chaining? >> >> Thank you! >> >>
