Re: [ceph-users] Getting "No space left on device" when reading from cephfs

2019-05-09 Thread Kári Bertilsson
Thanks for the tips A single OSD was indeed 95% full, and after removing it there is 24TB usable space and everything working again. :D I hope during the backfilling, another OSD won't go 95% also. It's a bit odd with ~140 OSD's that a single full one can take everything down with it. I would un

Re: [ceph-users] Getting "No space left on device" when reading from cephfs

2019-05-09 Thread Paul Emmerich
One full OSD stops everything. You can change what's considered 'full', the default is 95% ceph osd set-full-ratio 0.95 Never let an OSD run 100% full, that will lead to lots of real problems, 95% is a good default (it's not exact, some metadata might not always be accounted or it might temporar

Re: [ceph-users] Getting "No space left on device" when reading from cephfs

2019-05-09 Thread EDH - Manuel Rios Fernandez
. Regards De: ceph-users En nombre de Kári Bertilsson Enviado el: jueves, 9 de mayo de 2019 14:08 Para: ceph-users Asunto: [ceph-users] Getting "No space left on device" when reading from cephfs Hello I am running cephfs with 8/2 erasure coding. I had about 40tb usable free(

[ceph-users] Getting "No space left on device" when reading from cephfs

2019-05-09 Thread Kári Bertilsson
Hello I am running cephfs with 8/2 erasure coding. I had about 40tb usable free(110tb raw), one small disk crashed and i added 2x10tb disks. Now it's backfilling & recovering with 0B free and i can't read a single file from the file system... This happend with max-backfilling 4, but i have increa