[ceph-users] 14.2.2 -> 14.2.3 upgrade [WRN] failed to encode map e905 with expected crc

2019-09-06 Thread Stefan Kooman
Hi, While upgrading the monitors on a Nautilus test cluster warning messages apear: [WRN] failed to encode map e905 with expected crc Is this expected? I have only seen this in the past when mixing different releases (major versions), not when upgrading within a release. What is the impact of

[ceph-users] Automatic balancing vs supervised optimization

2019-09-06 Thread Massimo Sgaravatto
Hi I have question regarding supervised/automatic balancing using upmap. I created a plan in supervised mode, but its score was not expected to improve the data distribution. But the automatic balancer triggered a considerable rebalance. Is this normal ? I thought that automatic balancing basica

[ceph-users] regurlary 'no space left on device' when deleting on cephfs

2019-09-06 Thread Kenneth Waegeman
Hi all, We are using cephfs to make a copy of another fs via rsync, and also use snapshots. I'm seeing this issue now and then when I try to delete files on cephFS: |[root@osd001 ~]# rm -f /mnt/ceph/backups/osd00*|| ||rm: cannot remove ‘/mnt/ceph/backups/osd001.gigalith.os-3eea7740.1542483’:

Re: [ceph-users] regurlary 'no space left on device' when deleting on cephfs

2019-09-06 Thread Stefan Kooman
Quoting Kenneth Waegeman (kenneth.waege...@ugent.be): > The cluster is healthy at this moment, and we have certainly enough space > (see also osd df below) It's not well balanced though ... do you use ceph balancer (with balancer in upmap mode)? Gr. Stefan -- | BIT BV https://www.bit.nl/

Re: [ceph-users] regurlary 'no space left on device' when deleting on cephfs

2019-09-06 Thread Paul Emmerich
Yeah, no ENOSPC error code on deletion is a little bit unintuitive, but what it means is: the purge queue is full. You've already told the MDS to purge faster. Not sure how to tell it to increase the maximum backlog for deletes/purges, though, but you should be able to find something with the sear

[ceph-users] Ceph for "home lab" / hobbyist use?

2019-09-06 Thread William Ferrell
Hello everyone! After years of running several ZFS pools on a home server and several disk failures along the way, I've decided that my current home storage setup stinks. So far there hasn't been any data loss, but recovering/"resilvering" a ZFS pool after a disk failure is a nail-biting experienc

Re: [ceph-users] Ceph for "home lab" / hobbyist use?

2019-09-06 Thread Cranage, Steve
I use those HC2 nodes for my home Ceph cluster, but my setup only has to support the librados API, my software does HSM between regular XFS file systems and the RADOS api so I don’t need the other MDS and the rest so I can’t tell you if you’ll be happy in your configuration. Steve Cranage Pr

Re: [ceph-users] Ceph for "home lab" / hobbyist use?

2019-09-06 Thread Peter Woodman
2GB ram is gonna be really tight, probably. However, I do something similar at home with a bunch of rock64 4gb boards, and it works well. There are sometimes issues with the released ARM packages (frequently crc32 doesn;'t work, which isn't great), so you may have to build your own on the board you

Re: [ceph-users] Ceph for "home lab" / hobbyist use?

2019-09-06 Thread William Ferrell
On Fri, Sep 6, 2019 at 6:37 PM Peter Woodman wrote: > > 2GB ram is gonna be really tight, probably. Bummer. So it won't be enough for any Ceph component to run reliably? Drat. The nice thing about the HC2 is the fact that it can power the attached SATA disk and itself through one barrel connecto