Hello Gürkan, Indeed, by design, snapshots are only possible at the top of a subvolume (which is invisible in a pod). This is deliberate, as otherwise they would become a mechanism for quota evasion.
You can try CSI-level snapshots, which use cephfs-level snapshots under the hood. Or, you can even try giving your pod a permission to talk to the CSI driver, which would then make snapshots on its behalf, but I don't know offhand how to do it. In any case, please also try asking in Kubernetes forums. On Ceph side, unfortunately, everything works as intended. On Sat, Mar 1, 2025 at 2:58 AM <c...@gurkan.in> wrote: > > Hello all, > > We're getting a "Operation not permitted" error while trying to create a > snapshot on the client. It is somehow related to previously-asked Pacific > issue mentioned here: https://www.spinics.net/lists/ceph-users/msg67908.html > > We are on squid (19.2.1) and the given workaround seems to be only temporary. > Anyway, here are some details about the issue: > > Cephx side: > > ``` > client.mount-update-production > key: *** > caps: [mds] allow rws fsname=production-cephfs > path=/volumes/_nogroup/update-production > caps: [mon] allow r fsname=production-cephfs > caps: [osd] allow rw tag cephfs data=production-cephfs > ``` > > FS flags: > ``` > ~> sudo ceph fs lsflags production-cephfs > joinable allow_snaps allow_multimds_snaps > ``` > > I got the path via following command: > ``` > ~> sudo ceph fs subvolume getpath production-cephfs update-production > /volumes/_nogroup/update-production/e155ba30-d201-4653-be3f-86533324ee5f > ``` > > And mounted to client with mentioned key: > ``` > ~> mount | grep ceph > mount-update-production@00000000-0000-0000-0000-000000000000.production-cephfs=/volumes/_nogroup/update-production/e155ba30-d201-4653-be3f-86533324ee5f > on /opt/aptly/.aptly type ceph > (rw,noatime,name=mount-update-production,secret=<hidden>,acl,mon_addr=[multiple > mon addresses here]) > ``` > > Everything works flawlessly. Yet, when we want to get a snapshot: > ``` > /opt/aptly/.aptly/pool/9c/0b ~> sudo mkdir .snap/test > mkdir: cannot create directory ‘.snap/test’: Operation not permitted > ``` > > If I mount the whole FS with admin rights and run `setfattr -n > ceph.dir.subvolume -v 0` for _all_ directories of following path one by one: > /volumes/_nogroup/update-production/e155ba30-d201-4653-be3f-86533324ee5f, > then it is solving the issue. But it does not last; not sure what is > triggering it again, but if I try snapshotting anything next day, getting the > same "Operation not permitted" error again. > > Is there something I can try, or am I missing something obvious? > > Thanks, > Gürkan > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io -- Alexander Patrakov _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
