Hi,
Some osds of our cluster filled up:
health HEALTH_ERR 1 full osd(s); 4 near full osd(s)
monmap e1: 3 mons at
{ceph001=10.141.8.180:6789/0,ceph002=10.141.8.181:6789/0,ceph003=10.141.8.182:6789/0}, election epoch 96, quorum 0,1,2
ceph001,ceph002,ceph003
mdsmap e93: 1/1/1 up {0=ceph001.cubone.os=up:active}, 1 up:standby
osdmap e1974: 42 osds: 42 up, 42 in
flags full
pgmap v286626: 1200 pgs, 3 pools, 31096 GB data, 26259 kobjects
94270 GB used, 40874 GB / 131 TB avail
1 active+clean+scrubbing+deep
1199 active+clean
I knew it is never really uniform, but the differences of the OSDs
seems very big, one OSD has 96% while another only has 48% usage,
which is about 1,8 TB difference:
/dev/sdc 3.7T 1.9T 1.8T 51% /var/lib/ceph/osd/sdc
/dev/sdd 3.7T 2.5T 1.2T 68% /var/lib/ceph/osd/sdd
/dev/sde 3.7T 2.3T 1.5T 61% /var/lib/ceph/osd/sde
/dev/sdf 3.7T 2.7T 975G 74% /var/lib/ceph/osd/sdf
/dev/sdg 3.7T 3.2T 491G 87% /var/lib/ceph/osd/sdg
/dev/sdh 3.7T 2.0T 1.8T 53% /var/lib/ceph/osd/sdh
/dev/sdi 3.7T 2.3T 1.4T 63% /var/lib/ceph/osd/sdi
/dev/sdj 3.7T 3.4T 303G 92% /var/lib/ceph/osd/sdj
/dev/sdk 3.7T 2.8T 915G 76% /var/lib/ceph/osd/sdk
/dev/sdl 3.7T 1.8T 2.0T 48% /var/lib/ceph/osd/sdl
/dev/sdm 3.7T 2.8T 917G 76% /var/lib/ceph/osd/sdm
/dev/sdn 3.7T 3.5T 186G 96% /var/lib/ceph/osd/sdn
We are running 0.79 (well precisely a patched version of it with an
MDS fix of another thread:-) )
I remember hearing something about the hashpgpool having an effect on
it, but I read this was already default enabled on the latest
versions. osd_pool_default_flag_hashpspool has indeed the value true,
but I don't know how to check this for a specific pool.
Is this behaviour normal? Or what can be wrong?
Thanks!
Kind regards,
Kenneth Waegeman
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
