rp- opened a new pull request, #14344:
URL: https://github.com/apache/cloudstack/pull/14344
### Description
This PR fixes live migration with storage
(`migrateVirtualMachineWithVolume`) from another primary
storage (e.g. NFS) into a LINSTOR primary storage, which fails whenever the
LINSTOR auto-placement
does not put a copy of the new resource on the migration target host:
```
Failed to migrate VM [...] along with its volumes due to [...] failed in
LinstorDataMotionStrategy.copyAsync.
Error message: [Exception during migrate: org.libvirt.LibvirtException:
internal error:
cannot precreate storage for disk type 'block']
```
`LinstorDataMotionStrategy.copyAsync` spawns the destination resource from
the resource group, so
LINSTOR picks the nodes without knowing about the target host. The
`PrepareForMigrationCommand`
sent to the target still describes the source disks, so the target agent
never connects the new
volume either. If the target host has no copy,
`/dev/drbd/by-res/cs-<uuid>/0` does not exist there,
and libvirt can only precreate missing file disks, not block devices.
On 3-node clusters with 2 replicas this rarely shows up, because LINSTOR
adds a diskless quorum
tiebreaker on the third node. It fails with place count 1, with the
tiebreaker disabled, with 4+
nodes and 2 replicas, or when the resource group's filters exclude the
target host. Offline
migration (stop the VM, migrate the volume) is not affected, because
starting the VM goes through
`connectPhysicalDisk`.
Changes:
- After creating the destination resource, make it available on the target
host
(`resourceMakeAvailableOnNode`, a diskless resource for DRBD) and take the
device path from
there. No dual-primary is needed, the source disk is not this LINSTOR
resource.
- If the copy fails before the migrate command is sent (make-available,
prepare for migration),
the already created destination volumes are now destroyed and expunged.
Before, they were left
behind in LINSTOR and in the `volumes` table.
- The error message named the source host as "storage(s)"; it now lists the
destination pools.
- Linstor plugin CHANGELOG entry.
The strategy was added in #12532, so 4.20.3.0 and 4.22.1.0 are affected.
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] Build/CI
- [ ] Test (unit or integration test code)
### Feature/Enhancement Scale or Bug Severity
#### Feature/Enhancement Scale
- [ ] Major
- [ ] Minor
#### Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [x] Major
- [ ] Minor
- [ ] Trivial
### Screenshots (if appropriate):
### How Has This Been Tested?
3 KVM hosts (Ubuntu 24.04), LINSTOR 1.35.2 / DRBD 9.3, a 4.22.1.1-based
CloudStack build with this
patch on the management server (`LinstorDataMotionStrategy` is identical to
the 4.22 branch). To make the failure deterministic on 3 nodes, the target
LINSTOR pool uses a
resource group with place count 1 restricted to a storage pool that only
exists on host 2. A
running VM with its root disk on an NFS primary storage is migrated from
host 1 to host 3 with
`migrateVirtualMachineWithVolume`, root volume to the LINSTOR pool.
- Without the patch: fails with `cannot precreate storage for disk type
'block'` (reproduced).
- With the patch: migration succeeds, the VM keeps running (no reboot) on
host 3 from
`/dev/drbd/by-res/cs-<uuid>/0` (diskless, InUse; diskful copy on host 2
only), a 64 MiB random
file written in the guest before the migration has the same sha256
afterwards, the NFS volume
is expunged.
#### How did you try to break this feature and the system with this change?
- Make-available fails: LINSTOR satellite on the target host stopped. The
migration fails with the
LINSTOR error, the VM stays running on the source host on NFS, the
destination volume is
expunged and its LINSTOR resource definition is deleted.
- Migration fails after make-available: qemu migration/NBD ports
(49152-49215) rejected on the
target host. The migration fails in `blockdev-add`, the VM stays on the
source host, the
destination resource definition, including the new diskless resource on
the target, is deleted.
- Linstor plugin unit tests and checkstyle pass.
Not tested: shared (thick LVM) LINSTOR storage pools, VMs with multiple
volumes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
