hfutatzhanghb commented on PR #3743: URL: https://github.com/apache/hadoop/pull/3743#issuecomment-2111693845
> Not sure why this PR has been pick up again. IIRC we have discussed add tools to support refresh topology online many times. I would like to share my concern again. This is one high-risky operation, especially for one large cluster. Considering the case that change the cluster topology randomly then namenode failover or restart, there will be block transfer flood where the replications of one block should be going to have enough racks. Before all blocks have enough racks (which is cost long times) the cluster could enter high-load phase and couldn't process any requests from client. In one word, be careful to refresh topology online if there is no stability guarantee measures. @Hexiaoqiao Sir, thanks for your reminding. I got your concern here. I think this PR just provide a tool which can refresh topology without restarting namenode. As for how to use this tool safely to prevent causing transfer flood belongs to end-user. They can modify a small part of topologay_mappings.data every time, then refreshTopology and wait for transfering completed. Do this procedure until all parts of topologay_mappings.data updated finally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
