Improve namenode scalability by prioritizing datanode heartbeats over block
reports
-----------------------------------------------------------------------------------
Key: HDFS-1392
URL: https://issues.apache.org/jira/browse/HDFS-1392
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Reporter: dhruba borthakur
Assignee: dhruba borthakur
When a namenode restarts, it gets heartbeats followed by a block reports from
the datanodes. The block report processing is heavyweight and can take many
many seconds, especially when the number of blocks in each datanode is large.
This causes the namenode to not be able to process heartbeats in time, thus
leading to "lost heartbeats from datanode". The namenode declares the datanode
to be dead and the datanode has to re-send its register/block report all over
again. This is a scalability problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
