Hey Gyula, Is the Adaptive Scheduler limited to the Reactive mode? I agree that if we move forward with the Adaptive Scheduler solution it should support all deployment scenarios.
Thanks, -Max On Sun, Oct 9, 2022 at 6:10 AM Gyula Fóra <gyula.f...@gmail.com> wrote: > Hi! > > I think we have to make sure that the Rescale API will work also without > the adaptive scheduler (for instance when we are running Flink with the > Kubernetes Native Integration or in other cases where the adaptive > scheduler is not supported). > > What do you think? > > Cheers > Gyula > > > > On Fri, Oct 7, 2022 at 3:50 PM Maximilian Michels <m...@apache.org> wrote: > >> We've been looking into ways to support programmatic rescaling of job >> vertices. This feature is typically required for any type of Flink >> autoscaler which does not merely set the default parallelism but adjusts >> the parallelisms on a JobVertex level. >> >> We've had an initial discussion here: >> https://issues.apache.org/jira/browse/FLINK-29501 where Chesnay suggested >> to use the infamous "rescaling" API: >> >> https://nightlies.apache.org/flink/flink-docs-master/docs/ops/rest_api/#jobs-jobid-rescaling >> This API is disabled as of >> https://issues.apache.org/jira/browse/FLINK-12312 >> . >> >> Since there is the Adaptive Scheduler in Flink now, it may be feasible to >> re-enable the API (at least for streaming jobs) and allow overriding the >> parallelism of job vertices in addition to the default parallelism. >> >> Any thoughts? >> >> -Max >> >
