Hi! I think we have to make sure that the Rescale API will work also without the adaptive scheduler (for instance when we are running Flink with the Kubernetes Native Integration or in other cases where the adaptive scheduler is not supported).
What do you think? Cheers Gyula On Fri, Oct 7, 2022 at 3:50 PM Maximilian Michels <m...@apache.org> wrote: > We've been looking into ways to support programmatic rescaling of job > vertices. This feature is typically required for any type of Flink > autoscaler which does not merely set the default parallelism but adjusts > the parallelisms on a JobVertex level. > > We've had an initial discussion here: > https://issues.apache.org/jira/browse/FLINK-29501 where Chesnay suggested > to use the infamous "rescaling" API: > > https://nightlies.apache.org/flink/flink-docs-master/docs/ops/rest_api/#jobs-jobid-rescaling > This API is disabled as of > https://issues.apache.org/jira/browse/FLINK-12312 > . > > Since there is the Adaptive Scheduler in Flink now, it may be feasible to > re-enable the API (at least for streaming jobs) and allow overriding the > parallelism of job vertices in addition to the default parallelism. > > Any thoughts? > > -Max >
