Re: [VOTE] FLIP-56: Dynamic Slot Allocation

2019-09-24 Thread Xintong Song
Thanks all for the votes. So far, we have - 4 binding +1 votes (Till, Andrey, Gary and Kurt) - 1 un-binding +1 votes (Xintong) - No -1 votes There are more than 3 binding +1 votes and no -1 votes, and the voting time has past. According to the community bylaws, I'm glad to announce that

Re: [VOTE] FLIP-56: Dynamic Slot Allocation

2019-09-24 Thread Xintong Song
Thanks for the votes, Gary and Kurt. @Kurt Sorry for the confusion. I've added a clarification in the section "Unknown Resource Requirement". And +1 (non-binding) from my side. Thank you~ Xintong Song On Tue, Sep 24, 2019 at 5:35 PM Kurt Young wrote: > If it's possible, I would suggest to

Re: [VOTE] FLIP-56: Dynamic Slot Allocation

2019-09-24 Thread Kurt Young
If it's possible, I would suggest to add one sector in this doc to emphasize that current design has a prerequisite that each job should either has all its operators using unknown resource profile or all using specified amount of resource. This would make this document easier to understand. (I was

Re: [VOTE] FLIP-56: Dynamic Slot Allocation

2019-09-24 Thread Gary Yao
Hi Xintong, Thanks for starting the vote. The proposed changes look good to me. +1 (binding) Best, Gary On Mon, Sep 23, 2019 at 4:18 PM Till Rohrmann wrote: > Thanks for updating the Flip. It looks good to me. > > +1 (binding) > > Cheers, > Till > > On Mon, Sep 23, 2019 at 4:12 PM Xintong Son

Re: [VOTE] FLIP-56: Dynamic Slot Allocation

2019-09-23 Thread Till Rohrmann
Thanks for updating the Flip. It looks good to me. +1 (binding) Cheers, Till On Mon, Sep 23, 2019 at 4:12 PM Xintong Song wrote: > @Till @Andrey > > According to the comments, I just updated the FLIP document [1], with the > following changes: > >- Remove SlotID (in the section Protocol Ch

Re: [VOTE] FLIP-56: Dynamic Slot Allocation

2019-09-23 Thread Xintong Song
@Till @Andrey According to the comments, I just updated the FLIP document [1], with the following changes: - Remove SlotID (in the section Protocol Changes) - Updated implementation steps to reduce separated code paths. As far as I can see at the moment, we do not need the feature option

Re: [VOTE] FLIP-56: Dynamic Slot Allocation

2019-09-19 Thread Xintong Song
I'm not sure if I understand the implementation plan you suggested correctly. To my understanding, it seems that all the steps except for step 5 have to happen in strict order. - Profiles to be used in step 2 is reported with step 1. - SlotProfile in TaskExecutorGateway#requestSlot in step 3

Re: [VOTE] FLIP-56: Dynamic Slot Allocation

2019-09-19 Thread Till Rohrmann
I think besides of point 1. and 3. there are no dependencies between the RM and TM side changes. Also, I'm not sure whether it makes sense to split the slot manager changes up into the proposed steps 5, 6 and 7. I would highly recommend to not add too much duplicate logic/separate code paths becau

Re: [VOTE] FLIP-56: Dynamic Slot Allocation

2019-09-19 Thread Xintong Song
Thanks for the comments, Till. - Agree on removing SlotID. - Regarding the implementation plan, it is true that we can possibly reduce codes separated by the feature option. But I think to do that we need to introduce more dependencies between implementation steps. With the current plan, we can e

Re: [VOTE] FLIP-56: Dynamic Slot Allocation

2019-09-19 Thread Till Rohrmann
Hi Xintong, thanks for starting the vote. The general plan looks good. Hence +1 from my side. I still have some minor comments one could think about: * As we no longer have predetermined slots on the TaskExecutor, I think we can get rid of the SlotID. Instead, an allocated slot will be identified

Re: [VOTE] FLIP-56: Dynamic Slot Allocation

2019-09-19 Thread Andrey Zagrebin
Hi Xintong, Thanks for starting the vote, +1 from my side. Best, Andrey On Tue, Sep 17, 2019 at 4:26 PM Xintong Song wrote: > Hi all, > > I would like to start the vote for FLIP-56 [1], on which a consensus is > reached in this discussion thread [2]. > > The vote will be open for at least 72 h

[VOTE] FLIP-56: Dynamic Slot Allocation

2019-09-17 Thread Xintong Song
Hi all, I would like to start the vote for FLIP-56 [1], on which a consensus is reached in this discussion thread [2]. The vote will be open for at least 72 hours. I'll try to close it after Sep. 20 15:00 UTC, unless there is an objection or not enough votes. Thank you~ Xintong Song [1] https