Hi,
I haven't dealt with journal based mirroring in quite a while, but
this section [0] in the docs seems relevant:
rbd-mirror tunables are set by default to values suitable for
mirroring an entire pool. When using rbd-mirror to migrate single
volumes between clusters you may achieve substantial performance
gains by setting rbd_journal_max_payload_bytes=8388608 within the
[client] config section of the local or centralized configuration.
Note that this setting may allow rbd-mirror to present a substantial
write workload to the destination cluster: monitor cluster
performance closely during migrations and test carefully before
running multiple migrations in parallel.
# ceph config help rbd_journal_max_payload_bytes
rbd_journal_max_payload_bytes - maximum journal payload size before splitting
(size, advanced)
Default: 16384
Can update at runtime: true
Services: [rbd]
Regards,
Eugen
[0]
https://docs.ceph.com/en/latest/rbd/rbd-mirroring/#enable-image-journaling-feature
Zitat von "GLE, Vivien" <[email protected]>:
Hi,
I'm testing mirroring in ceph and dealt with very slow mirroring operation,
Cluster A (primary) is on a Ceph hyperconverged on proxmox and
cluster B (secondary) is a Ceph cluster managed by cephadm. These 2
have been peered via bootstrap.
Moving 60 GiB VM to the mirrored pool in cluster A it took 5-6
hours to fully sync in the cluster B (SSD device only)
Moving 60 GiB VM from cluster A to cluster B in replica 3 SSD pool
took 2 minutes (SSD device only)
It looks like for this purpose cluster wont be higher than 6-7
MiB/s, is this a normal behaviour ?
Ceph -s during mirroring output and rbd info below
ceph -s
cluster:
id: ID
health: HEALTH_OK
services:
mon: 3 daemons, quorum r620-13-1,r620-13-9,r620-13-4 (age 12d)
mgr: r620-13-7.zxzajo(active, since 24h), standbys:
r620-13-10.wmzodp
mds: 1/1 daemons up
osd: 24 osds: 24 up (since 6d), 24 in (since 6d)
rbd-mirror: 1 daemon active (1 hosts)
rgw: 1 daemon active (1 hosts, 1 zones)
data:
volumes: 1/1 healthy
pools: 16 pools, 1287 pgs
objects: 151.21k objects, 551 GiB
usage: 1.6 TiB used, 6.3 TiB / 7.9 TiB avail
pgs: 1287 active+clean
io:
client: 15 KiB/s rd, 6.5 MiB/s wr, 11 op/s rd, 419 op/s wr
--------------------------------------------
rbd info ceph_proxmox/vm-118-disk-0
rbd image 'vm-118-disk-0':
size 60 GiB in 15360 objects
order 22 (4 MiB objects)
snapshot_count: 0
id: abe8c560f69498
block_name_prefix: rbd_data.abe8c560f69498
format: 2
features: layering, exclusive-lock, object-map, fast-diff,
deep-flatten, journaling
op_features:
flags:
create_timestamp: Wed Oct 22 07:52:31 2025
access_timestamp: Wed Oct 22 07:52:31 2025
modify_timestamp: Wed Oct 22 07:52:31 2025
journal: abe8c560f69498
mirroring state: enabled
mirroring mode: journal
mirroring global id: id
mirroring primary: false
-----------------------------------------
In cluster A ceph.conf
rbd_default_features = 125
Thanks !
Vivien
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
