Dear All,
du (and quota) report that files are using double the apparent-size, we
suspect this is due to DOM being enabled.
This is a new cluster, running 2.16-1 with 3 x MDT and 7 x OST, using
ZFS for both MDT and OST
looking at a couple of files created with dd:
fmb384_gollum> du -sh *
734M big_file
390G huge_file
compare to:
fmb384_gollum> du -sh --apparent-size *
319M big_file
170G huge_file
DOM is enabled:
root@dino-s1 11:22 [~]: lfs getstripe /scratch/gollum/huge_file
/scratch/gollum/huge_file
lcm_layout_gen: 3
lcm_mirror_count: 1
lcm_entry_count: 2
lcme_id: 1
lcme_mirror_id: 0
lcme_flags: init
lcme_extent.e_start: 0
lcme_extent.e_end: 1048576
lmm_stripe_count: 0
lmm_stripe_size: 1048576
lmm_pattern: mdt
lmm_layout_gen: 0
lmm_stripe_offset: 0
lcme_id: 2
lcme_mirror_id: 0
lcme_flags: init
lcme_extent.e_start: 1048576
lcme_extent.e_end: EOF
lmm_stripe_count: 1
lmm_stripe_size: 1048576
lmm_pattern: raid0
lmm_layout_gen: 0
lmm_stripe_offset: 3
lmm_objects:
- 0: { l_ost_idx: 3, l_fid: [0x380000402:0xa8e72:0x0] }
Whereas folder without DOM:
# lfs setstripe /scratch/jog/noDOM
# dd if=/dev/zero of=noDOM_100M_file bs=1M count=100
# du -h noDOM_100M_file
92M noDOM_100M_file
compare to:
# du -h --apparent-size noDOM_100M_file
100M noDOM_100M_file
quota agrees:
root@dino-s1 14:13 [scratch]: lfs quota -h -u jog
Disk quotas for usr jog (uid 1022):
Filesystem used quota limit grace files quota
limit grace
/scratch 92.05M 0k 0k - 4 0
0
Quota agrees with du, unless we use apparent-size:
fmb384_gollum> du -h /scratch/gollum/*
229M /scratch/gollum/DOM_100M_file
fmb384_gollum> du -h --apparent-size /scratch/gollum/*
100M /scratch/gollum/DOM_100M_file
fmb384_gollum> lfs quota -h -u gollum /scratch
Disk quotas for usr gollum (uid 1084):
Filesystem used quota limit grace files quota limit
grace
/scratch 228.3M 1.5T 2T - 5 0 0
-
Any thoughts gratefully received...
Jake
--
Dr Jake Grimmett
Head Of Scientific Computing
MRC Laboratory of Molecular Biology
Francis Crick Avenue, Cambridge CB2 0QH, UK.
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
