Re: FSImage size issue

2020-12-01 Thread Jason Wen
The heap histo does show it is caused by snapshot 2: 36069513725970049864 org.apache.hadoop.hdfs.server.namenode.INodeFileAttributes$SnapshotCopy 3: 36069513720198927672 org.apache.hadoop.hdfs.server.namenode.snapshot.FileDiff 4: 36345766311630645216 org.apache.hadoop.

Re: FSImage size issue

2020-12-01 Thread Jason Wen
I also suspect it might be caused by snapshots. We don't have lots of xattr or ALCs on our files. How can I confirm if it is really caused by snapshots? Thanks, Jason On 12/1/20, 4:43 PM, "Wei-Chiu Chuang" wrote: Possibly snapshots. Also possible if each of your files has lots of xatt

Re: FSImage size issue

2020-12-01 Thread Wei-Chiu Chuang
Possibly snapshots. Also possible if each of your files has lots of xattr or ACLs. On Tue, Dec 1, 2020 at 3:52 PM Jason Wen wrote: > Hi, > > We are encountering some odd FSImage size issue in one of our Hadoop > clusters. The Namenode only has about 3M files/blocks, but the FSImage size > is abo