Hi Ted,
Yes, it is safe to bounce the HMaster without taking the region servers down.
Are there any regions in transition (would be shown on the master's page)?
All of the threads involved with
-
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(org.apache.hadoop.fs.Path)
@bci=14, line=309 (Compiled frame)
...
- org.apache.hadoop.hbase.master.HMaster.getHTableDescriptors(java.util.List)
@bci=50, line=1932 (Compiled frame)
are a bit suspect. Who's requesting all the table descriptors from the master?
-- Lars
From: Ted Tuttle <[email protected]>
To: "[email protected]" <[email protected]>
Cc: Development <[email protected]>
Sent: Thursday, March 26, 2015 12:18 PM
Subject: master consumes large amount of CPU for days
Hello-
Our master process started consuming a large amount of CPU (75% of box) several
days back and hasn't stopped. I have 2 questions:
1) what is it doing (stack dump and log below)
2) is it safe to restart the master without taking the whole
cluster down?
Master stack dump:
http://pastebin.com/G0iNNEpC
Master log from last 15 mins or so:
http://pastebin.com/WQNjhFGf
Thanks,
Ted
