Hi! I think what you need is probably not the reactive mode but a proper autoscaler. The reactive mode as you say doesn't do anything in itself, you need to build a lot of logic around it.
Check this instead: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/custom-resource/autoscaler/ The Kubernetes Operator has a built in autoscaler that can scale jobs based on kafka data rate / processing throughput. It also doesn't rely on the reactive mode. Cheers, Gyula On Fri, Aug 18, 2023 at 12:43 PM Dennis Jung <inylov...@gmail.com> wrote: > Hello, > Sorry for frequent questions. This is a question about 'reactive' mode. > > 1. As far as I understand, though I've setup `scheduler-mode: reactive`, > it will not change parallelism automatically by itself, by CPU usage or > Kafka consumer rate. It needs additional resource monitor features (such as > Horizontal Pod Autoscaler, or else). Is this correct? > 2. Is it possible to create a custom resource monitor provider > application? For example, if I want to increase/decrease parallelism by > Kafka consumer rate, do I need to send specific API from outside, to order > rescaling? > 3. If 2 is correct, what is the difference when using 'reactive' mode? > Because as far as I think, calling a specific API will rescale either using > 'reactive' mode or not...(or is the API just working based on this mode)? > > Thanks. > > Regards > >
