Re: [Question] How to scale application based on 'reactive' mode

Also, the metrics is on a per-task granularity and allows us to identify >>>bottleneck tasks. >>>3. Autoscaler feature currently only works for K8s opeartor + native >>>K8s mode. >>> >>> >>> Best, >>> Zhanghao Chen >>&

Re: [Question] How to scale application based on 'reactive' mode

;Also, the metrics is on a per-task granularity and allows us to identify >>bottleneck tasks. >>3. Autoscaler feature currently only works for K8s opeartor + native >>K8s mode. >> >> >> Best, >> Zhanghao Chen >> -----

Re: [Question] How to scale application based on 'reactive' mode

k tasks. >3. Autoscaler feature currently only works for K8s opeartor + native >K8s mode. > > > Best, > Zhanghao Chen > -- > *发件人:* Dennis Jung > *发送时间:* 2023年9月2日 12:58 > *收件人:* Gyula Fóra > *抄送:* user@flink.apache.org > *主题:*

回复: [Question] How to scale application based on 'reactive' mode

us to identify bottleneck tasks. 3. Autoscaler feature currently only works for K8s opeartor + native K8s mode. Best, Zhanghao Chen 发件人: Dennis Jung 发送时间: 2023年9月2日 12:58 收件人: Gyula Fóra 抄送: user@flink.apache.org 主题: Re: [Question] How to scale application

Re: [Question] How to scale application based on 'reactive' mode

Hello, Thanks for your notice. 1. In "Flink 1.18 + non-reactive", is parallelism being changed by the number of TM? 2. In the document( https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.6/docs/custom-resource/autoscaler/), it said "we are not using any container memory /

Re: [Question] How to scale application based on 'reactive' mode

Pretty much, except that with Flink 1.18 autoscaler can scale the job in place without restarting the JM (even without reactive mode ) So actually best option is autoscaler with Flink 1.18 native mode (no reactive) Gyula On Fri, 1 Sep 2023 at 13:54, Dennis Jung wrote: > Thanks for feedback. >

Re: [Question] How to scale application based on 'reactive' mode

Thanks for feedback. Could you check whether I understand correctly? *Only using 'reactive' mode:* By manually adding TaskManager(TM) (such as using './bin/taskmanager.sh start'), parallelism will be increased. For example, when job parallelism is 1 and TM is 1, and if adding 1 new TM, JobManager

Re: [Question] How to scale application based on 'reactive' mode

I would look at reactive scaling as a way to increase / decrease parallelism. It’s not a way to automatically decide when to actually do it as you need to create new TMs . The autoscaler could use reactive mode to change the parallelism but you need the autoscaler itself to decide when new resour

Re: [Question] How to scale application based on 'reactive' mode

For now, the thing I've found about 'reactive' mode is that it automatically adjusts 'job parallelism' when TaskManager is increased/decreased. https://www.slideshare.net/FlinkForward/autoscaling-flink-with-reactive-mode Is there some other feature that only 'reactive' mode offers for scaling? T

Re: [Question] How to scale application based on 'reactive' mode

Hello, Thank you for your response. I have few more questions in following: https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/deployment/elastic_scaling/ *Reactive Mode configures a job so that it always uses all resources available in the cluster. Adding a TaskManager will scale up

Re: [Question] How to scale application based on 'reactive' mode

The reactive mode reacts to available resources. The autoscaler reacts to changing load and processing capacity and adjusts resources. Completely different concepts and applicability. Most people want the autoscaler , but this is a recent feature and is specific to the k8s operator at the moment.

Re: [Question] How to scale application based on 'reactive' mode

Hello, Thanks for your notice. Than what is the purpose of using 'reactive', if this doesn't do anything itself? What is the difference if I use auto-scaler without 'reactive' mode? Regards, Jung 2023년 8월 18일 (금) 오후 7:51, Gyula Fóra 님이 작성: > Hi! > > I think what you need is probably not the r

Re: [Question] How to scale application based on 'reactive' mode

Hi! I think what you need is probably not the reactive mode but a proper autoscaler. The reactive mode as you say doesn't do anything in itself, you need to build a lot of logic around it. Check this instead: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/custom-resou

[Question] How to scale application based on 'reactive' mode

Hello, Sorry for frequent questions. This is a question about 'reactive' mode. 1. As far as I understand, though I've setup `scheduler-mode: reactive`, it will not change parallelism automatically by itself, by CPU usage or Kafka consumer rate. It needs additional resource monitor features (such a