Re: [VOTE] FLIP-461: Synchronize rescaling with checkpoint creation to minimize reprocessing for the AdaptiveScheduler

+1 (non-binding) Best, Zdenek On Mon, Jun 17, 2024 at 10:24 AM Matthias Pohl wrote: > Hi everyone, > the discussion in [1] about FLIP-461 [2] is kind of concluded. I am > starting a vote on this one here. > > The vote will be open for at least 72 hours (i.e. until June 20, 2024; > 8:30am UTC) u

[DISCUSS] FLIP-XXX: Aligning timeout logic in the AdaptiveScheduler's WaitingForResources and Executing states

more detailed information, please refer to the FLIP document. https://docs.google.com/document/d/1YeYSs64LqgUr3xyBTCjiRE-CT5VEyHjGjqxnxKPIQhM/edit?usp=sharing Thanks, Zdenek Tison

Re: [DISCUSS] FLIP-XXX: Aligning timeout logic in the AdaptiveScheduler's WaitingForResources and Executing states

onfigure the rescaling behavior. For example, if a user sets the lower bound parallelism to 2 and the upper bound to 3, the system will rescale after 10 seconds. Alternatively, if the user sets the same value for the lower and upper bounds, the rescale operation will wait until all slots are availa

Re: [DISCUSS] FLIP-XXX: Aligning timeout logic in the AdaptiveScheduler's WaitingForResources and Executing states

-scheduler-resource-stabilization-timeout > [5] > > https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#jobmanager-adaptive-scheduler-max-delay-for-scale-trigger > > > > On Tue, Jul 16, 2024 at 3:05 PM Zdenek Tison > wrote: > > > Hi, I

Re: [DISCUSS] FLIP-XXX: Aligning timeout logic in the AdaptiveScheduler's WaitingForResources and Executing states

lization, otherwise if > you receive a batch of new resources you may not utilize it because as soon > as you have sufficient we rescale immediately. > > Cheers, > Gyula > > > > On Thu, Jul 18, 2024 at 9:58 AM Zdenek Tison > wrote: > > > Thanks, Mathias, for you

Re: [DISCUSS] FLIP-XXX: Aligning timeout logic in the AdaptiveScheduler's WaitingForResources and Executing states

xyBTCjiRE-CT5VEyHjGjqxnxKPIQhM/edit> Thanks a lot. Regards, Zdenek On Wed, Jul 24, 2024 at 2:22 PM Zdenek Tison wrote: > Hi Gyula, > > Thank you for reviewing the document and providing feedback. > >1. I agree that we need two separate parameters for stabilization >

Re: [DISCUSS] FLIP-XXX: Aligning timeout logic in the AdaptiveScheduler's WaitingForResources and Executing states

Hi, If there are no further comments, I would propose starting a vote on these changes. But first, I would like to ask a committer to migrate the draft to an FLIP in the Flink Wiki. Thanks a lot. Kind Regards, Zdenek On Tue, Jul 30, 2024 at 10:36 AM Zdenek Tison wrote: > Hi all, > &

Re: [DISCUSS] FLIP-XXX: Aligning timeout logic in the AdaptiveScheduler's WaitingForResources and Executing states

Hi Rui, Yes, it's right. We will update the FLIP's parameters table to make it more explicit. Thanks Best, Zdenek On Mon, Aug 5, 2024 at 5:33 AM Rui Fan <1996fan...@gmail.com> wrote: > Thanks Zdenek Tison and Mattias for driving this proposal! > It's indeed a gre

[VOTE] FLIP-472: Aligning timeout logic in the AdaptiveScheduler's WaitingForResources and Executing states

Hi everyone, Thanks for all the feedback about FLIP-472: Aligning timeout logic in the AdaptiveScheduler's WaitingForResources and Executing states [1]. The discussion thread can be found here [2]. The vote will be open for at least 72 hours unless there are any objections or insufficient vote

Re: [DISCUSS] FLIP-XXX: Aligning timeout logic in the AdaptiveScheduler's WaitingForResources and Executing states

[1]. > > [1] > > https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=315494203&selectedPageVersions=7&selectedPageVersions=5 > > On Mon, Aug 5, 2024 at 8:56 AM Zdenek Tison > wrote: > > > Hi Rui, > > > > Yes, it's right. >

[RESULT][VOTE] FLIP-472: Aligning timeout logic in the AdaptiveScheduler's WaitingForResources and Executing states

e.org/confluence/display/FLINK/FLIP-472%3A+Aligning+timeout+logic+in+the+AdaptiveScheduler%27s+WaitingForResources+and+Executing+states Best, Zdenek Tison

[jira] [Created] (FLINK-30403) The reported latest completed checkpoint is discarded

Zdenek Tison created FLINK-30403: Summary: The reported latest completed checkpoint is discarded Key: FLINK-30403 URL: https://issues.apache.org/jira/browse/FLINK-30403 Project: Flink Issue

[jira] [Created] (FLINK-36011) Generalize RescaleManager to become StateTransitionManager

Zdenek Tison created FLINK-36011: Summary: Generalize RescaleManager to become StateTransitionManager Key: FLINK-36011 URL: https://issues.apache.org/jira/browse/FLINK-36011 Project: Flink

[jira] [Created] (FLINK-36012) Integrate StateTransitionManager into WaitingForResources state

Zdenek Tison created FLINK-36012: Summary: Integrate StateTransitionManager into WaitingForResources state Key: FLINK-36012 URL: https://issues.apache.org/jira/browse/FLINK-36012 Project: Flink

[jira] [Created] (FLINK-36013) Introduce the transition from Restarting to CreatingExecutionGraph state

Zdenek Tison created FLINK-36013: Summary: Introduce the transition from Restarting to CreatingExecutionGraph state Key: FLINK-36013 URL: https://issues.apache.org/jira/browse/FLINK-36013 Project

[jira] [Created] (FLINK-36014) Align the desired and sufficient resources definiton in Executing and WaitForResources states

Zdenek Tison created FLINK-36014: Summary: Align the desired and sufficient resources definiton in Executing and WaitForResources states Key: FLINK-36014 URL: https://issues.apache.org/jira/browse/FLINK-36014

[jira] [Created] (FLINK-36015) Align rescale parameters

Zdenek Tison created FLINK-36015: Summary: Align rescale parameters Key: FLINK-36015 URL: https://issues.apache.org/jira/browse/FLINK-36015 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-36016) Synchronize initialization time and clock usage

Zdenek Tison created FLINK-36016: Summary: Synchronize initialization time and clock usage Key: FLINK-36016 URL: https://issues.apache.org/jira/browse/FLINK-36016 Project: Flink Issue Type