Hi! So, OpenZFS is adding reflink support Soon(tm), including across filesystems on a pool, which is nice.
Unfortunately, Linux's VFS returns EXDEV for trying FICLONE or FICLONERANGE (but not copy_file_range) cross-filesystem before you ever ask the filesystem-specific code, so currently, the following strange behavior occurs: On coreutils 8.30 or 8.32, cp --reflink=always across filesystems will fail with EXDEV and --reflink=auto will not reflink (because they're not trying copy_file_range as a fallback). On coreutils git, as of b3331d59e, cp --reflink=always across filesystems will fail with EXDEV without ever getting out of Linux's VFS code, cp --reflink=auto will reflink silently (since it falls back to copy_file_range after getting EXDEV), cp --reflink=never will not reflink. (On the same filesystem, in all of the above versions, cp --reflink=always and =auto do the same thing and reflink correctly.) I'm not sure what the "correct" behavior here should be, but at least =auto working and =always failing seems like a surprising and incorrect outcome to me, though it's not readily obvious to me how the code "should" flow instead to avoid that - and since the failure cases happen before calling into OpenZFS, I don't see any way it could be handled better there. Happy to point people at the WIP code being used to demonstrate this if it would be helpful, but this seems like it's only OpenZFS specific in that nobody else has this functionality but would hit this case (because IIUC btrfs avoids clone_file failing with EXDEV by pretending they're not distinct filesystems, and there's not many other FSes where reflink across filesystems would make sense). Thanks for any insights, - Rich
