Yuanbo Liu created HDFS-17538:
---------------------------------
Summary: Add tranfer priority queue for decommissioning datanode
Key: HDFS-17538
URL: https://issues.apache.org/jira/browse/HDFS-17538
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Yuanbo Liu
Attachments: image-2024-05-29-16-24-45-601.png,
image-2024-05-29-16-26-58-359.png, image-2024-05-29-16-27-35-886.png
When decommissioning datanode, blocks will be checked one by one disk, then
blocks will be sent to trigger tranfer works in DN. This will make one disk of
decommissioning dn very busy and cpus stuck in io-wait with high loads, and
sometime even lead to OOM as below:
!image-2024-05-29-16-24-45-601.png!
!image-2024-05-29-16-26-58-359.png!
!image-2024-05-29-16-27-35-886.png!
Proposal to add priority queue for transfering blocks when decommisioning
datanode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
