[
https://issues.apache.org/jira/browse/YARN-11669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Teke resolved YARN-11669.
----------------------------------
Fix Version/s: 3.5.0
Hadoop Flags: Reviewed
Resolution: Fixed
> [Umbrella] cgroup v2 support
> ----------------------------
>
> Key: YARN-11669
> URL: https://issues.apache.org/jira/browse/YARN-11669
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: yarn
> Reporter: Ferenc Erdelyi
> Assignee: Benjamin Teke
> Priority: Major
> Fix For: 3.5.0
>
>
> cgroup v2 has some fundamental changes compared to v1. RHEL9, Ubuntu 22
> already moved to cgroup v2 as a default, hence YARN should support it. This
> umbrella tracks the required work.
> [Documentation|https://docs.kernel.org/admin-guide/cgroup-v2.html]
> A way to test the newly added features:
> # Turn on cgroup v1 based on the current
> [documentation|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeManagerCgroups.html].
> # System prerequisites:
> ## the file {{/etc/mtab}} should contain a mount path with the file system
> type {{cgroup2}}, by default this could be {{/sys/fs/cgroup}} on most OS's
> ## the {{cgroup.subtree_control}} file should contain the necessary
> controllers (update it with: {{echo "+cpu +io +memory" >
> cgroup.subtree_control}})
> ## either create the YARN hierarchy and give recursive access to the user
> running the NM on the node. The hierarchy is {{hadoop-yarn}} by default
> (controller by
> {{yarn.nodemanager.linux-container-executor.cgroups.hierarchy}}), and
> recursive mode is required, because as soon as the directory is created it
> will be filled with the controller files which YARN will try to edit.
> ### Alternatively if the NM process user has access rights on the
> {{/sys/fs/cgroup}} directory it'll try to create the hierarchy and update the
> {{cgroup.subtree_control}} file.
> # YARN configuration
> ## {{yarn.nodemanager.linux-container-executor.cgroups.mount-path}} should
> point to the directory where the cgroup2 structure is mounted and the
> {{hadoop-yarn}} hierarchy was created
> ## {{yarn.nodemanager.linux-container-executor.cgroups.v2.enabled}} should be
> set to {{true}}
> ## Enable a cgroup controller, like {{yarn. nodemanager. resource.
> cpu.enabled}}: {{true}}
> # Launch the NM and monitor the cgroup files on container launches (i.e:
> {{/sys/fs/cgroup/hadoop-yarn/container_id/cpu.weight}})
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
