On Thu, Mar 26, 2020 at 9:13 AM Frank Schilder <fr...@dtu.dk> wrote: > > Dear all, > > yes, this is it, quotas. In the structure A/B/ there was a quota set on A. > Hence, B was moved out of this zone and this does indeed change mv to be a > cp+rm. > > The obvious follow-up. What is the procedure for properly moving data as an > administrator? Do I really need to unset quotas, do the move and set quotas > back again?
Ah-ha, this is a difference between the userspace and kernel clients. :( The kernel client always returns EXDEV if crossing "quota realms" (different quota roots). I'm not sure why it behaves that way as userspace is different: * If fhere is a quota in the target directory, and * If the target directory's existing data, plus the file(s) being moved, exceed the target directory's quota, then userspace returns EXDEV/EDQUOT. This seems like the right kernel behavior as well... Jeff, is that a known issue for some reason? Should we make a new bug? :) -Greg > > Best regards, > > ================= > Frank Schilder > AIT Risø Campus > Bygning 109, rum S14 > > ________________________________________ > From: Frank Schilder <fr...@dtu.dk> > Sent: 26 March 2020 16:35:07 > To: Gregory Farnum; Beocat KSU > Cc: ceph-users > Subject: [ceph-users] Re: Move on cephfs not O(1)? > > Dear all, thanks for the hints. I was afraid I see ghosts. > > Yes, we have quotas on many directories. I'm not entirely sure if there were > different quotas involved. I believe it was set only on the highest level, > but now that you mention it ... > > I will try with quotas again, didn't include this in my scenario. > > This information is really important, because it will affect how I as an > admin can move data between folders with quotas on them, which are typically > used for sub-directory mounts. For a user this would look like a magical move > between file systems. > > I will get back. Thanks! > > Best regards, > > ================= > Frank Schilder > AIT Risø Campus > Bygning 109, rum S14 > > ________________________________________ > From: Gregory Farnum <gfar...@redhat.com> > Sent: 26 March 2020 15:20:51 > To: Beocat KSU > Cc: ceph-users; Frank Schilder > Subject: Re: [ceph-users] Re: Move on cephfs not O(1)? > > I was wondering that but at least in the userspace client the quotas > will just throw EXDEV or EDQUOT if it will exceed quotas... > ...and EXDEV might trigger the kernel to do a copy-and-delete, I > guess? Not sure. > > On Thu, Mar 26, 2020 at 7:18 AM Adam Tygart <mo...@ksu.edu> wrote: > > > > Is there is a possibility that there was a quota involved? I've seen > > moves between quota zones to cause a copy then delete. > > > > -- > > Adam > > > > On Thu, Mar 26, 2020 at 9:14 AM Gregory Farnum <gfar...@redhat.com> wrote: > > > > > > On Thu, Mar 26, 2020 at 5:49 AM Frank Schilder <fr...@dtu.dk> wrote: > > > > > > > > Some time ago I made a surprising observation. I reorganised a > > > > directory structure and needed to move a folder one level up with a > > > > command like > > > > > > > > mv A/B/ B > > > > > > > > B contained something like 9TB in very large files. To my surprise, > > > > this command didn't return for a couple of minutes and I started to > > > > look what was going on. What I discovered was, that the mv command > > > > actually performed a full copy with a subsequent remove. I had to wait > > > > for several hours for the move to complete. > > > > > > > > I tried to reproduce this today to collect further information. > > > > However, this behaviour seems not reproducible. No matter what I try, > > > > mv completes almost instantly. > > > > > > > > I was running the original mv on mimic 13.2.2 and retried now with > > > > mimic 13.2.8. In addition, there was an OS upgrade from Centos 7.6 to > > > > 7.7. I'm using the kernel-ml versions (5.xxx). Only one cephfs mount > > > > was present at all times. > > > > > > > > My questions are: > > > > > > > > 1) Was there a change from 13.2.2 to 13.2.8 explaining this? > > > > > > No. > > > > > > > 2) Are there (rare) conditions under which an mv on cephfs becomes a > > > > cp+rm? > > > > > > Not as part of CephFS. > > > > > > > 3) Am I seeing ghosts? > > > > > > The only way I can imagine this happening is if you had separate > > > CephFS mounts for cephfs/A/B and cephfs/ and you did the move between > > > them, instead of within the cephfs/ mount. :/ > > > -Greg > > > > > > > > > > > Thanks for clues and best regards, > > > > > > > > ================= > > > > Frank Schilder > > > > AIT Risø Campus > > > > Bygning 109, rum S14 > > > > _______________________________________________ > > > > ceph-users mailing list -- ceph-users@ceph.io > > > > To unsubscribe send an email to ceph-users-le...@ceph.io > > > > > > > _______________________________________________ > > > ceph-users mailing list -- ceph-users@ceph.io > > > To unsubscribe send an email to ceph-users-le...@ceph.io > > > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
