Hi Gyula, Could I ask if the autoscaler will support the native session cluster on 1.18 flink operator? Also, please correct me if I'm wrong. The autoscale will work similarly to the elastic scaling in reactive mode, right? For example, in the K8s cluster, if one pod is added to the session cluster, the job running on will be rebalanced to the new one, is it correct?
Thank you very much. Xiao Ma On Wed, Feb 1, 2023 at 10:56 AM Gyula Fóra <gyula.f...@gmail.com> wrote: > As I mentioned in the previous email, standalone mode is not on the > Autoscaler roadmap because the scheduling/resource model is different. > This applies to both standalone app and session clusters. > > Thanks > Gyula > > On Wed, Feb 1, 2023 at 4:48 PM Swathi Chandrashekar <cswa...@microsoft.com> > wrote: > >> Sure, thanks Gyula. >> Is there a roadmap to support standalone session clusters to scale based >> on the jobs added/deleted and change in parallelism ? >> >> Regards, >> Swathi C >> >> ------------------------------ >> *From:* Gyula Fóra <gyula.f...@gmail.com> >> *Sent:* Wednesday, February 1, 2023 8:54 PM >> *To:* Swathi Chandrashekar <cswa...@microsoft.com> >> *Cc:* user@flink.apache.org <user@flink.apache.org> >> *Subject:* [EXTERNAL] Re: Query on flink-operator autoscale support >> >> The autoscaler currently only works with Native App clusters. >> Native session clusters may be supported in the future but standalone is >> not on our roadmap due to a different resource/scheduling model used. >> >> Gyula >> >> On Wed, Feb 1, 2023 at 4:22 PM Swathi Chandrashekar < >> cswa...@microsoft.com> wrote: >> >> Hi, >> >> I'm was testing OSS flink operator with flink 1.17 for autoscale feature. >> I was able to scale the cluster based on load in application cluster in >> native mode, but the same did not work in standalone mode as the operator >> gave the following error as below [ both for app and session mode ]. >> >> Is the autoscale supported for the following : >> >> 1. Session cluster in standalone >> 2. Session cluster in native >> 3. App cluster in standalone >> >> >> Exception in thread "pool-4-thread-3" java.lang.NoSuchMethodError: >> org.apache.flink.kubernetes.operator.kubeclient.Fabric8FlinkStandaloneKubeClient.create(Lorg/apache/flink/configuration/Configuration;Ljava/util/concurrent/ExecutorService;)Lorg/apache/flink/kubernetes/operator/kubeclient/Fabric8FlinkStandaloneKubeClient; >> at >> org.apache.flink.kubernetes.operator.service.StandaloneFlinkService.createNamespacedKubeClient(StandaloneFlinkService.java:105) >> at >> org.apache.flink.kubernetes.operator.service.StandaloneFlinkService.submitClusterInternal(StandaloneFlinkService.java:110) >> at >> org.apache.flink.kubernetes.operator.service.StandaloneFlinkService.deployApplicationCluster(StandaloneFlinkService.java:69) >> at >> org.apache.flink.kubernetes.operator.service.AbstractFlinkService.submitApplicationCluster(AbstractFlinkService.java:180) >> at >> org.apache.flink.kubernetes.operator.reconciler.deployment.ApplicationReconciler.deploy(ApplicationReconciler.java:175) >> at >> org.apache.flink.kubernetes.operator.reconciler.deployment.ApplicationReconciler.deploy(ApplicationReconciler.java:58) >> at >> org.apache.flink.kubernetes.operator.reconciler.deployment.AbstractFlinkResourceReconciler.reconcile(AbstractFlinkResourceReconciler.java:115) >> at >> org.apache.flink.kubernetes.operator.controller.FlinkDeploymentController.reconcile(FlinkDeploymentController.java:130) >> at >> org.apache.flink.kubernetes.operator.controller.FlinkDeploymentController.reconcile(FlinkDeploymentController.java:56) >> at >> io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:136) >> at >> io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:94) >> at >> org.apache.flink.kubernetes.operator.metrics.OperatorJosdkMetrics.timeControllerExecution(OperatorJosdkMetrics.java:80) >> at >> io.javaoperatorsdk.operator.processing.Controller.reconcile(Controller.java:93) >> at >> io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.reconcileExecution(ReconciliationDispatcher.java:130) >> at >> io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleReconcile(ReconciliationDispatcher.java:110) >> at >> io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleDispatch(ReconciliationDispatcher.java:81) >> at >> io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:54) >> at >> io.javaoperatorsdk.operator.processing.event.EventProcessor$ReconcilerExecutor.run(EventProcessor.java:406) >> at >> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) >> at >> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) >> at java.base/java.lang.Thread.run(Thread.java:834) >> >> >> Regards, >> Swathi C >> >>
