Ming Ma created HDFS-8875:
-----------------------------
Summary: Optimize the wait time in Balancer for federation scenario
Key: HDFS-8875
URL: https://issues.apache.org/jira/browse/HDFS-8875
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Ming Ma
Assignee: Chris Trezzo
Balancer has wait time between two consecutive iterations. That is to give some
time for block movement to be fully committed ( return from replaceBlock
doesn't mean the NN's blockmap has been updated and the block has been
invalidated on the source node.).
This wait time could be 23 seconds if {{dfs.heartbeat.interval}} is set to 10
and {{dfs.namenode.replication.interval}} is to 3. In the case of federation,
given we iterate through all namespaces in each iteration, this wait time
becomes unnecessary as while balancer is processing the next namespace, it
gives the previous namespace it just finished time to commit.
In addition, Balancer calls {{Collections.shuffle(connectors);}} It doesn't
seem necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
