Re: [DISCUSS] FLIP-495: Support AdaptiveScheduler record and query the rescale history

2025-03-24 Thread Yuepeng Pan
Hi, Community, There haven’t been any further responses to this email over the past few days. I'd like to initiate a vote on the current proposal[1] in the next few days. Please rest assured that I’m proceeding cautiously and not rushing the process. If there are any concerns about this FLIP-495[1

Re: Re: [DISCUSS] FLIP-495: Support AdaptiveScheduler record and query the rescale history

2025-03-11 Thread Yuepeng Pan
Hi, Matthias, So sorry for the long delay in completing and following up. I'm still making the necessary research to address the issues mentioned previously in this FLIP. FYI, the new round of adjustments has been completed, and I've attached an overview of a few issues. > Also keep in mind th

Re: Re: [DISCUSS] FLIP-495: Support AdaptiveScheduler record and query the rescale history

2025-01-12 Thread Matthias Pohl
Hi Yupeng, I managed to find some time to respond. See my answers inlined below. Matthias On Fri, Jan 3, 2025 at 11:48 AM Yuepeng Pan wrote: > [...] > Sorry for not expressing this part clearly earlier. > IIUC, based on the Adaptive Scheduler state diagram [1], > when a stop-with-savepoint oper

Re:Re: [DISCUSS] FLIP-495: Support AdaptiveScheduler record and query the rescale history

2025-01-03 Thread Yuepeng Pan
Thanks Matthias very much for the sharing and suggestions. The ideas make sense to me on the whole and I'd be willing to update it. I just want to confirm a few questions before starting the next round of wiki adjustments: 1 - About the Boundary Definition of Rescale Events > A few ot

Re: [DISCUSS] FLIP-495: Support AdaptiveScheduler record and query the rescale history

2025-01-02 Thread Matthias Pohl
Thanks Yuepeng for your response. I added my comments to the individual paragraphs below: > Thank you very much for the reminding The proposal makes sense to me. > Additionally, I'd like to confirm whether each rescale cycle/event > requires a status field, such as FAILED, IGNORED, SUCCESS, PENDI

Re: [DISCUSS] FLIP-495: Support AdaptiveScheduler record and query the rescale history

2024-12-19 Thread Yuepeng Pan
Thanks Matthias for the comments. >- How is the resourceRequirementsEpochID generated? Please let me have a try on clarifying it. Each rescale requirement corresponds to a resourceRequirementsEpochID. A new value(UUID) will be generated for resourceRequirementsEpochID when the job is first star

Re: [DISCUSS] FLIP-495: Support AdaptiveScheduler record and query the rescale history

2024-12-18 Thread Matthias Pohl
Hi Yuepeng, Sorry for not finding the time to respond earlier. I went over FLIP-495 [1] and the previous FLIP-487 discussion [2]. Thanks for putting it all together in a FLIP. That makes it easier to discuss the next iteration. Here are a few comments I have: Rescale ID section - How is the resour

[DISCUSS] FLIP-495: Support AdaptiveScheduler record and query the rescale history

2024-12-17 Thread Yuepeng Pan
Hi community, We discussed several aspects of FLIP-487[1] 'Show history of rescales in Web UI for AdaptiveScheduler' and received a lot of valuable feedback. Based on the suggestions from the email thread[2], we plan to split the original proposal for FLIP-487[1]. The current email thre