Syed Shameerur Rahman created YARN-11752:
--------------------------------------------
Summary: Global Scheduler : Improve the container allocation time
Key: YARN-11752
URL: https://issues.apache.org/jira/browse/YARN-11752
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Syed Shameerur Rahman
Assignee: Syed Shameerur Rahman
As part of YARN-10380, The allocation strategy for multi-mode/global scheduler
was changed from "node" list based to "partition" based, Due to this change,
After every two container allocation, the scheduler waits for *5 milli seconds*
by default as mentioned
[here|[https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java#L627].]
This significantly affects the container throughput. YARN SLS benchmark shows
Async scheduling was *6x* better than Global scheduling in terms of container
throughput for scheduling 1600 container with 50 nodes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
