got it, you can use it as a service and see what will happen.
---- Replied Message ----
| From | Sébastien Rebecchi<srebec...@kameleoon.com> |
| Date | 03/09/2025 22:22 |
| To | Zhanghaobo<hfutzhan...@163.com> |
| Cc | u...@hadoop.apache.org、hdfs-dev@hadoop.apache.org |
| Subject | Re: Can not run HDFS balancer cause metrics already exists |
Hi Zhanghaobo,
Thanks for the message.
No I don't use as service, as I said the command line is the following: hdfs
balancer -Ddfs.balancer.movedWinWidth=5400000 -Ddfs.balancer.moverThreads=1000
-Ddfs.balancer.dispatcherThreads=200
-Ddfs.datanode.balance.max.concurrent.moves=50
-Ddfs.datanode.balance.bandwidthPerSec=100m
-Ddfs.balancer.max-size-to-move=10737418240 -threshold 1
Also no other balancer is running concurrently on any other node.
Sébastien
Le dim. 9 mars 2025 à 13:57, Zhanghaobo <hfutzhan...@163.com> a écrit :
Hi, @Sébastien Rebecchi
Don't know more details about how you start balancer, did you use -asService?
---- Replied Message ----
| From | Sébastien Rebecchi<srebec...@kameleoon.com.INVALID> |
| Date | 3/9/2025 18:03 |
| To | <u...@hadoop.apache.org>,
<hdfs-dev@hadoop.apache.org> |
| Subject | Re: Can not run HDFS balancer cause metrics already exists |
Hello
Could anyone help on this please?
Situation is still the same after several days.
I add some precisions
- hadoop version 3.4.1
- balancer command line run: hdfs balancer -Ddfs.balancer.movedWinWidth=5400000
-Ddfs.balancer.moverThreads=1000 -Ddfs.balancer.dispatcherThreads=200
-Ddfs.datanode.balance.max.concurrent.moves=50
-Ddfs.datanode.balance.bandwidthPerSec=100m
-Ddfs.balancer.max-size-to-move=10737418240 -threshold 1
Thank you
Le mar. 4 mars 2025, 16:59, Sébastien Rebecchi <srebec...@kameleoon.com> a
écrit :
Hello
After having added a new node on my HDFS cluster, I try running balancer, but
it always fails with the following error, even after retrying multiple times
during the day, and even after having restarted name node
What should I do to unlock?
Thanks,
Sébastien
ERROR balancer.Balancer: Exiting balancer due an exception
org.apache.hadoop.metrics2.MetricsException: Metrics source Balancer-{HERE
REPLACE BY CLUSTER'S BLOCK POOL ID} already exists!
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:152)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:125)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:229)
at
org.apache.hadoop.hdfs.server.balancer.BalancerMetrics.create(BalancerMetrics.java:52)
at
org.apache.hadoop.hdfs.server.balancer.Balancer.<init>(Balancer.java:362)
at
org.apache.hadoop.hdfs.server.balancer.Balancer.doBalance(Balancer.java:824)
at
org.apache.hadoop.hdfs.server.balancer.Balancer.run(Balancer.java:868)
at
org.apache.hadoop.hdfs.server.balancer.Balancer$Cli.run(Balancer.java:975)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:82)
at
org.apache.hadoop.hdfs.server.balancer.Balancer.main(Balancer.java:1133)
