Just try to combine the discussion into one thread. @Diljeet Singh Posted a quick PoC for the proposal https://github.com/apache/flink-kubernetes-operator/pull/1020.
On Mon, Aug 25, 2025 at 7:52 AM Peter Huang <huangzhenqiu0...@gmail.com> wrote: > Hi Community, > > Our org has been heavily using the Flink autoscaling algorithm. It greatly > reduced our operation overhead and improved cost efficiency > as users always over provision resources when onboard. Recently, we have > had some requirements to customize the auto scaling algorithm > for different scenarios, for example, during the holiday season large but > predictable traffic spike, increase checkpoint interval together with > scale up for streaming ingestion use cases. > > We search through the discussion about the topic in the mail list > including the existing FLIP-514 > <https://cwiki.apache.org/confluence/display/FLINK/FLIP-514%3A+Custom+Evaluator+plugin+for+Flink+Autoscaler>. > Looks like the discussion is not finalized yet. > To accelerate the process, we adopt and combine the existing opinions from > the community and create a proposal in FLIP-543 > <https://cwiki.apache.org/confluence/display/FLINK/FLIP-543%3A+Support+Customized+Autoscale+Algorithm>. > The basic idea > is to make some core components of autoscaler pluggable, for example, > MetricsCollector, Metrics Evaluator, and ScalingRealizer, at the same > keep the core logic skeleton (which is already well justified in large > amount of users) of autoscaler untouched. > > Looking forward to any feedback and opinions on FLIP-543. > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-543%3A+Support+Customized+Autoscale+Algorithm > [2] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-514%3A+Custom+Evaluator+plugin+for+Flink+Autoscaler > [3] Other related discussion thread > > https://lists.apache.org/thread/749l74z1h5jylkxrw3rtjmxcj2t9p7ws > > https://lists.apache.org/thread/mcd7jcn4kz6oqtyqq5hfycjf9mqh6c53 > > > Best Regards > Peter Huang >
