Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-20 Thread Hangxiang Yu
gt; Thanks Roman for driving this useful improvement, +1 for this > > proposal. > > >> >> > > >> >> Also thanks discussion from Hangxiang and Rui Fan. Regarding > question > > >> 1, I > > >> >> have some ideas for discuss

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-20 Thread Gustavo de Morais
t; Also thanks discussion from Hangxiang and Rui Fan. Regarding question > >> 1, I > >> >> have some ideas for discussion: > >> >> > >> >> Based on the consideration of providing stable expectations for > users, > >> I > >> &

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-19 Thread Roman Khachatryan
should perform configuration checks in a whitelist manner. >> Ensure >> >> that the configurations allowed to be modified through this API can >> >> actually >> >> take effect. >> >> >> >> In the initial version, we can provide a ve

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-17 Thread Roman Khachatryan
initial version, we can provide a very small whitelist list, > even if > >> it only contains a few configurations that we most want to use and have > >> been > >> confirmed to be effective. This list can be continuously supplemented > >> later. > >> &g

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-15 Thread Kartikey Pant
Hi Roman, I've been following the FLIP-530 proposal for dynamic job configuration with great interest - thank you for spearheading this valuable initiative. The ability to adjust configurations on-the-fly will undoubtedly be a significant enhancement for Flink. As I was thinking through its pract

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-15 Thread Chesnay Schepler
st regards, Feifan Wang Replied Message | From | Rui Fan<1996fan...@gmail.com> | | Date | 05/11/2025 16:36 | | To | | | Subject | Re: [DISCUSS] FLIP-530: Dynamic job configuration | Thanks Roman for driving this valuable proposal, it uses the Adaptive Scheduler to greatly reduce the downtime

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-14 Thread Roman Khachatryan
Thanks all for the feedback! @ Rui Fan These are very good points, I agree. In the PoC, we've actually implemented validation of some specific options that can't be changed (in addition to the white list). I will look at the FLIPs you referred to. @Feifan Wang, Junrui Lee W.r.t the white-list, I'

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-13 Thread Andrei Kaigorodov
> > > > —————————— > > > > Best regards, > > Feifan Wang > > > > > > > > Replied Message > > | From | Rui Fan<1996fan...@gmail.com> | > > | Date | 05/11/2025 16:36 | > > | To | | > > | Subject | Re

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-11 Thread Junrui Lee
is list can be continuously supplemented > later. > > > —— > > Best regards, > Feifan Wang > > > > Replied Message > | From | Rui Fan<1996fan...@gmail.com> | > | Date | 05/11/2025 16:36 | > | To | | > | Subject | Re: [DISCUSS] F

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-11 Thread Feifan Wang
. This list can be continuously supplemented later. —— Best regards, Feifan Wang Replied Message | From | Rui Fan<1996fan...@gmail.com> | | Date | 05/11/2025 16:36 | | To | | | Subject | Re: [DISCUSS] FLIP-530: Dynamic job configuration | Thanks Roman for drivin

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-11 Thread Rui Fan
Thanks Roman for driving this valuable proposal, it uses the Adaptive Scheduler to greatly reduce the downtime of configuration updates, so +1 for this proposal! Overall LGTM, thanks to Hangxiang for the questions, and I have the same questions with Hangxiang. I'd like to share my thoughts: For

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-10 Thread Roman Khachatryan
Thanks Hangxiang Yu, Please find the answers below 1. Yes, we should perform validation before trying to update the configuration. I'd rather validate some specific options that are known to not work though. Finding and hard-coding all the valid options might be impractical since they can change,

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-08 Thread Hangxiang Yu
Hi, Roman. Thanks for the FLIP. +1 for supporting dynamic configuration to reduce manual restart. I just have below questions: 1. Do we need a working configuration list ? So some unsupported configurations could be rejected in advance. 2. Could we show the change history in the Web UI ? So mo