Flink job and task manager pods auto scaling

2025-05-19 Thread Kamal Mittal via user
Hello, Couple of questions below for Flink 1.20, please give input. 1. While using "Adaptive scheduler" for streaming job auto scaling, will the complete job re-start due to new parallelism? Any special situation? 1. While using "Adaptive scheduler" for streaming job auto scaling as wel

Unsubscribe

2025-05-19 Thread naga sudhakar
Unsubscribe

Re: Apache Flink Serialization Question

2025-05-19 Thread Zhanghao Chen
It would still work 发件人: Richard Cheung 发送时间: 星期二, 五月 20, 2025 4:08:00 上午 收件人: Zhanghao Chen 抄送: Мосин Николай ; Schwalbe Matthias ; user@flink.apache.org 主题: Re: Apache Flink Serialization Question Hi all, Thanks again for the help! I have one more follow u

Re: Apache Flink Serialization Question

2025-05-19 Thread Richard Cheung
Hi all, Thanks again for the help! I have one more follow up question regarding Flink and serialization on v1.18. I know state schema evolution is supported for POJOs in Flink. However, if my class uses the POJO serializer but has a field that falls back to Kryo (such as UUID), would it still be e

Re: Table API/SQL CDC Join produces deletes

2025-05-19 Thread Andrey
Hi Xu, Thank you for providing more context. I was using inner and left outer joins in my stream processing. After digging further I have understood that both D/UB and I/UA records are needed to handle many-to-one joins correctly in a general case when parallelisation can be greater than 1, and F

Re: Flink task manager pod auto scaling

2025-05-19 Thread Zhanghao Chen
For CPU scaling, you can do it by kill-and-restart or K8s VPA (beta in recent versions), and the algorithm should be straightforward. For MEM scaling, it is a bit challenging due to the complex memory model of Flink and the complexity of JVM itself. Flink K8s Operator provides AutoTuning for thi

Unsubscribe

2025-05-19 Thread Nadia Mujeeb
Thanks