Just wanted to bring this to the attention of the community due to the large performance delta we observing with the new patch (30-70x in some cases). I’ve tried to keep the changes as minimal as possible, but there are some refactorings done to allow the sorting code to be more localized. This is (IMO) a simplified approach vs. YUNIKORN-21 and hopefully carries less risk. Any review or comments would be greatly appreciated.
Thanks, Craig Condit > On Aug 19, 2021, at 10:03 AM, Craig Condit (Jira) <j...@apache.org> wrote: > > Craig Condit created YUNIKORN-807: > ------------------------------------- > > Summary: Improve performance of node sorting > Key: YUNIKORN-807 > URL: https://issues.apache.org/jira/browse/YUNIKORN-807 > Project: Apache YuniKorn > Issue Type: Improvement > Components: core - scheduler > Reporter: Craig Condit > Assignee: Craig Condit > Attachments: Node Sorting Performance Improvement.pdf > > YuniKorn currently sorts all nodes on demand whenever scheduling of a > container occurs. This causes significant performance degradation as the > number of nodes increases. > > If we replace the on-demand sorting with a B-Tree sorted proactively, we can > improve performance considerably. > > This is a similar approach to YUNIKORN-21, but without the associated > behavioral changes. > > I've attached a design document with the details of the approach and the > performance improvement gained. > > > > -- > This message was sent by Atlassian Jira > (v8.3.4#803005) > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org > For additional commands, e-mail: dev-h...@yunikorn.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
