Blocks in newly completed files are considered under-replicated too quickly
---------------------------------------------------------------------------
Key: HDFS-1172
URL: https://issues.apache.org/jira/browse/HDFS-1172
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 0.21.0
Reporter: Todd Lipcon
I've seen this for a long time, and imagine it's a known issue, but couldn't
find an existing JIRA. It often happens that we see the NN schedule replication
on the last block of files very quickly after they're completed, before the
other DNs in the pipeline have a chance to report the new block. This results
in a lot of extra replication work on the cluster, as we replicate the block
and then end up with multiple excess replicas which are very quickly deleted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
