On my test cluster I ran into the problem when creating the 2nd or third Ceph
FS, that the actual mounting and adding to the PVE storage config failed with
the following in the task log:
-----
creating data pool 'foobar_data'...
pool foobar_data: applying application = cephfs
pool foobar_data: applying pg_num = 32
creating metadata pool 'foobar_metadata'...
pool foobar_metadata: applying pg_num = 8
configuring new CephFS 'foobar'
Successfully create CephFS 'foobar'
Adding 'foobar' to storage configuration...
TASK ERROR: adding storage for CephFS 'foobar' failed, check log and add manually! create
storage failed: mount error: Job failed. See "journalctl -xe" for details.
------
The matching syslog:
------
Oct 20 15:20:04 cephtest1 systemd[1]: Mounting /mnt/pve/foobar...
Oct 20 15:20:04 cephtest1 mount[45484]: mount error: no mds server is up or the
cluster is laggy
Oct 20 15:20:04 cephtest1 systemd[1]: mnt-pve-foobar.mount: Mount process
exited, code=exited, status=32/n/a
Oct 20 15:20:04 cephtest1 systemd[1]: mnt-pve-foobar.mount: Failed with result
'exit-code'.
Oct 20 15:20:04 cephtest1 systemd[1]: Failed to mount /mnt/pve/foobar.
------
Adding the storage manually right after this worked fine. Seems like the MDS
might not be fast enough all the time.
Regarding the removal of a Ceph FS we had an off list discussion which resulted
in the following (I hope I am not forgetting something):
The process needs a few manual steps that are hard to automate:
- disable storage (so pvestatd does not auto mount it again)
- unmount on all nodes
- stop standby and active (for this storage) MDS
At this point, any still existing mount will be hanging
- remove storage cfg and pools
Since at least some of those need to be done manually on the CLI, it might not even be
worth it to have a "remove button" in the GUI but rather a well documented
procedure in the manual and the actual removal as part of `pveceph`.
On 10/19/21 11:33, Dominik Csapak wrote:
this series support for multiple cephfs. no single patch fixes the bug,
so it's in no commit subject... (feel free to change the commit subject
when applying if you find one patch most appropriate?)
a user already can create multiple cephfs via 'pveceph' (or manually
with the ceph tools), but the ui does not support it properly
storage patch can be applied independently, it only adds a new parameter
that does nothing if not set.
manager:
patches 1,2 enables basic gui support for showing correct info
for multiple cephfs
patches 3,4,5 are mostly preparation for the following patches
(though 4 enables some additional checks that should not hurt either way)
patch 6 enables additional gui support for multiple fs
patch 7,8 depend on the storage patch
patch 9,10,11 are for actually creating multiple cephfs via the gui
so those can be left out if we do not want to support that
---
so if we only want to support basic display functionality, we could only apply
manager 1,2 & maybe 5+6
for being able to configure multiple cephfs on a ceph cluster, we'd need
storage 1/1 and manager 7,8
sorry that it's so complicated, if wanted, i can ofc reorder the patches
or send it in multiple series
pve-storage:
Dominik Csapak (1):
cephfs: add support for multiple ceph filesystems
PVE/Storage/CephFSPlugin.pm | 8 ++++++++
1 file changed, 8 insertions(+)
pve-manager:
Dominik Csapak (11):
api: ceph-mds: get mds state when multple ceph filesystems exist
ui: ceph: catch missing version for service list
api: cephfs: refactor {ls,create}_fs
api: cephfs: more checks on fs create
ui: ceph/ServiceList: refactor controller out
ui: ceph/fs: show fs for active mds
api: cephfs: add 'fs-name' for cephfs storage
ui: storage/cephfs: make ceph fs selectable
ui: ceph/fs: allow creating multiple cephfs
api: cephfs: add destroy cephfs api call
ui: ceph/fs: allow destroying cephfs
PVE/API2/Ceph/FS.pm | 148 +++++++++--
PVE/Ceph/Services.pm | 16 +-
PVE/Ceph/Tools.pm | 51 ++++
www/manager6/Makefile | 2 +
www/manager6/Utils.js | 1 +
www/manager6/ceph/FS.js | 52 +++-
www/manager6/ceph/ServiceList.js | 313 ++++++++++++-----------
www/manager6/form/CephFSSelector.js | 42 +++
www/manager6/storage/CephFSEdit.js | 25 ++
www/manager6/window/SafeDestroyCephFS.js | 22 ++
10 files changed, 492 insertions(+), 180 deletions(-)
create mode 100644 www/manager6/form/CephFSSelector.js
create mode 100644 www/manager6/window/SafeDestroyCephFS.js
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
