Hi Akash, Flink doesn't support auto scaling in core currently, it may be supported in the future, when the new scheduling architecture is implemented https://issues.apache.org/jira/browse/FLINK-10407 .
You can do it externally by cancel the job with a savepoint, update the parallelism, and restart the job, according to the rate of data. like what pravega suggests in the doc: http://pravega.io/docs/latest/key-features/#auto-scaling. vino yang <yanghua1...@gmail.com> 于2019年11月29日周五 上午11:12写道： > Hi Akash, > > You can use Pravega connector to integrate with Flink, the source code is > here[1]. > > In short, relying on its rescalable state feature[2] flink supports > scalable streaming jobs. > > Currently, the mainstream solution about auto-scaling is Flink + K8S, I > can share some resources with you[3]. > > Best, > Vino > > [1]: https://github.com/pravega/flink-connectors > [2]: > https://flink.apache.org/features/2017/07/04/flink-rescalable-state.html > [3]: > https://www.slideshare.net/FlinkForward/flink-forward-san-francisco-2019-scaling-a-realtime-streaming-warehouse-with-apache-flink-parquet-and-kubernetes-aditi-verma-ramesh-shanmugam > > Akash Goel <akash.d.g...@gmail.com> 于2019年11月29日周五 上午9:52写道： > >> Hi, >> >> Does Flunk support auto scaling. I read that it is supported using >> pravega? Is it incorporated in any version. >> >> Thanks, >> Akash Goel >> >
