On 8/11/25 5:55 PM, Peter Xu wrote:
On Mon, Aug 11, 2025 at 05:26:05PM -0400, Jonah Palmer wrote:
This effort was started to reduce the guest visible downtime by
virtio-net/vhost-net/vhost-vDPA during live migration, especially
vhost-vDPA.
The downtime contributed by vhost-vDPA, for example, is not from having to
migrate a lot of state but rather expensive backend control-plane latency
like CVQ configurations (e.g. MQ queue pairs, RSS, MAC/VLAN filters, offload
settings, MTU, etc.). Doing this requires kernel/HW NIC operations which
dominates its downtime.
In other words, by migrating the state of virtio-net early (before the
stop-and-copy phase), we can also start staging backend configurations,
which is the main contributor of downtime when migrating a vhost-vDPA
device.
I apologize if this series gives the impression that we're migrating a lot
of data here. It's more along the lines of moving control-plane latency out
of the stop-and-copy phase.
I see, thanks.
Please add these into the cover letter of the next post. IMHO it's
extremely important information to explain the real goal of this work. I
bet it is not expected for most people when reading the current cover
letter.
Then it could have nothing to do with iterative phase, am I right?
What are the data needed for the dest QEMU to start staging backend
configurations to the HWs underneath? Does dest QEMU already have them in
the cmdlines?
Asking this because I want to know whether it can be done completely
without src QEMU at all, e.g. when dest QEMU starts.
If src QEMU's data is still needed, please also first consider providing
such facility using an "early VMSD" if it is ever possible: feel free to
refer to commit 3b95a71b22827d26178.
So the data to be transferred is still in VMSD form, aka, data are still
described by VMSD macros, instead of hard-coded streamline protocols using
e.g. qemufile APIs using save_setup()/load_setup().
When things are described in VMSDs, it get the most benefit from the live
migration framework, and it's much, much more flexible. It's the most
suggested way for device to cooperate with live migration, savevmhandlers
are only the last resort because it's almost not in control of migration..
In short, please avoid using savevmhandlers as long as there can be any
other way to achieve similar results.
Oh this early VMSD is interesting and, at first glance, appears to be
suitable for what we're trying to do here. I'll take a look at it and
see if this is something we can use instead of the SaveVMHandlers hooks.
Thank you for mentioning this.
Jonah
Thanks,
