DaanHoogland commented on code in PR #13656: URL: https://github.com/apache/cloudstack/pull/13656#discussion_r3915820732
########## docs/vmware-linstor-migration.md: ########## @@ -0,0 +1,238 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Linstor primary storage as a destination for VMware-to-KVM migration + +This document describes the Linstor-destination portion of the VMware-to-KVM +migration work, in enough detail for LINSTOR/DRBD maintainers to review the +design choices and advise on Linstor-side behavior. + +## Summary + +This change lets Linstor (DRBD) primary storage be the **destination** for +VMware-to-KVM instance imports, across all three migration modes CloudStack +offers for Ceph/RBD: + +- **Cold, staged** — convert with virt-v2v to a temporary NFS location, then + copy the finalized disks into Linstor. +- **Cold, direct (VDDK)** — read each disk over nbdkit/VDDK straight into the + DRBD device and finalize it in place, with no NFS staging. +- **Warm (CBT)** — replicate the running VM with an initial full sync plus + incremental changed-block cycles into the DRBD device, then finalize in + place at cutover. + +Before this change Linstor was blocked everywhere in the import/conversion +path by NFS-only (and, more recently, RBD-only) assumptions. The work +generalizes those code paths rather than special-casing Linstor: a new +`RAW_BLOCK_DEVICE` target type covers "write RAW into a host-local block +device provided by the storage adaptor", which also lays the groundwork for +other block backends (PowerFlex, FiberChannel, StorPool) later. + +## Background and motivation + +CloudStack can import VMware VMs onto KVM by converting them with virt-v2v +(optionally reading disks over the VMware VDDK). The converted disks have to +land on KVM primary storage. That path historically supported only +filesystem-style pools (NFS/Filesystem/SharedMountPoint); the companion +commits in this series add Ceph/RBD as a destination and a full VMware +**CBT warm-migration** framework. Linstor — a very common CloudStack block +backend — was still unsupported as a conversion destination. + +Linstor differs from RBD in ways that shaped the design: + +- A Linstor volume is a **local block device** (`/dev/drbd/by-res/<res>/0`), + which is the universal abstraction the whole toolchain (qemu-img, qemu-io, + virt-v2v-in-place) already consumes — so finalization is actually *simpler* + than RBD (no qemu-nbd credential bridge needed; a plain + `<disk type='block'>` works). +- Unlike RBD, `qemu-img convert` **cannot create** the device — the resource + must be pre-created at the source capacity through the storage adaptor, + then written with `qemu-img convert -n` (no-create). +- LINSTOR resource names are capped at ~48 characters and cannot be renamed, + so the RBD-style naming carrying a full migration UUID does not fit; short, + deterministic names are used and imported volumes keep the name as their + recorded volume path. + +## What this change does + +### New generic block-device target type +`VmwareCbtTargetStorageType.RAW_BLOCK_DEVICE`, alongside the existing +`QCOW2_FILE` and `RBD_RAW`. Linstor pools classify to it and require in-place +finalization (the qcow2 fallback cannot write to a device). + +### Server-side (orchestration) +- Linstor added to the staged-conversion destination pool types, the direct + VDDK-convert allow-list, and the CBT-compatible pool types, for both + explicit and implicit destination-pool selection. +- Conversion/import host selection validates that the host is a LINSTOR + satellite connected to the pool (in addition to the VDDK / in-place + virt-v2v capability checks), for auto-selection and explicit host IDs. +- `CheckConvertInstanceCommand` carries a new in-place-finalization capability + check so unsupported hosts fail fast with a clear message. +- The powered-off, non-cloned source requirement (already enforced for direct + RBD) now applies to any direct block-storage import. + +### KVM agent-side (data plane) +- **Staged**: the converted-disk move already dispatches per pool type to + `LinstorStorageAdaptor.copyPhysicalDisk` (qemu-img into the DRBD device); + converted-disk metadata for Linstor reports the pool UUID and volume name + instead of parsing an NFS mount. +- **Direct VDDK**: pre-creates each resource at source capacity via the + adaptor, then `nbdkit vddk --run 'qemu-img convert -n ... <device>'`, and + finalizes with `virt-v2v-in-place` fed a `<disk type='block'>` domain XML. +- **Warm CBT**: initial sync pre-creates the device and `qemu-img convert -n` + into it; delta cycles patch changed extents; cutover finalizes in place + with block-device XML (no qemu-nbd bridges). Cleanup deletes only + marker-guarded volumes. +- `LinstorStorageAdaptor.listPhysicalDisks` is implemented (resource + definitions filtered by the pool's resource group), replacing the previous + `UnsupportedOperationException`, so the forced-conversion import short + circuit works for Linstor. + +### Performance: nbdcopy for full-disk copies to block devices +The cold direct-VDDK import and the CBT initial full sync copy an entire disk +from the nbdkit/VDDK source into the target. For a local raw block-device +target (Linstor/DRBD), this now uses `nbdcopy` (libnbd) when the host has it, +falling back to `qemu-img convert`. nbdcopy keeps many requests in flight +(it is what virt-v2v itself uses for this step) and is typically faster than a +single-connection `qemu-img convert`. Availability is probed locally on the +agent; it is a pure optimization with automatic fallback, so no server-side +gating is needed. Only block-device targets are affected — the RBD URI target +and qcow2 file targets keep qemu-img convert (nbdcopy is raw-only and cannot +address an rbd: URI without a bridge). On a 16 GiB disk over network NBD +transport it measured ~20% faster than qemu-img convert in the lab, and the +result was byte-identical to a full VDDK read of the source. + +### Performance: direct CBT delta copy for block devices +Previously a CBT delta copied each changed extent in two hops — qemu-img the +nbd source window into a temporary raw file, then qemu-io write that file into +the target. For a raw block-device target that doubles local I/O and needs +scratch space. This change streams each changed extent from the nbd source +window straight into the same device window in a single +`qemu-img convert -n -S 0 --image-opts --target-image-opts`. `-S 0` disables +zero/sparse skipping so blocks the source cleared to zero are actually +overwritten in the target — a full copy can skip zeros because the target +starts zeroed, but a delta into an already-populated device cannot. The +qcow2-file and RBD paths are unchanged. + +## Migration support matrix (Linstor destination) + +| Mode | Trigger | Source VM state | Data movement | Guest downtime | +|---|---|---|---|---| Review Comment: @andrijapanicsb , a (small) markdown lint issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
