On 10/23/2024 12:31 PM, Paolo Bonzini wrote:
On 10/17/24 17:14, Steve Sistare wrote:
Define a new qemu initialization phase called 'precreate' which occurs
before most backends or devices have been created. The only exception
is monitor and qtest devices and their associated chardevs.
QEMU runs in the main loop during this phase. Monitor connections are
active and can receive migration configuration commands. QEMU starts
listening on the normal migration URI during this phase, which can come
from either the QEMU command line or from a migrate_incoming command.
Thus the user can issue query-migrate to get the socket-address for
dynamically allocated port numbers during precreate.
In this series QEMU passes through and does not linger in the precreate
phase, and the user sees no change in behavior. The cpr-transfer series
will linger in the phase for an incoming CPR operation, and exit the phase
when the migrate command is send to source QEMU and causes destination QEMU
to read CPR state.
A future series may wish to add a command-line argument and monitor
command that enters and exits the precreate phase for general purpose use.
That would enable the user to issue monitor commands that specify backend
creation parameters.
I have a problem with the concept, which is that the split between command line
and monitor is much harder to understand.
Ideally, one would come entirely before the other; preconfig is already ugly in
how -device is processed later than everything else[1]. This series makes this
much more complex.
If I understand correctly, what you want is effectively to execute monitor commands from
the migration stream. If you want to do that, we need to take more qemu_init code and
turn it into QMP commands, so that precreate can exit qemu_init very early and the
"after precreate" command line options can be replaced by interactions on the
monitor.
This is the idea that drove the creation of -M smp.xxx, -M boot.xxx, -M memory.xxx, etc.
(see for example commit 8c4da4b5218, "machine: add boot compound property",
2022-05-12). For example -semihosting-config, -acpitable, -smbios, -fw_cfg, -option-rom,
-rtc could all become -M properties and handled by a single monitor command machine-set.
Of all the other options, -accel, -cpu and -display are the main missing ones
(-cpu is the very hard one); a full list is at
https://wiki.qemu.org/User:Paolo_Bonzini/Machine_init_sequence#QMP_configuration_flow.
Anyhow, at this point all that's needed is a -chardev/-mon pair (and I guess
-incoming defer) in order to bootstrap the monitor in precreate mode.
It's okay to prototype without full support for the options I've listed, but if
we want to go with precreate we should make most command line options entirely
incompatible with it, and also make it imply -nodefaults.
Paolo
[1] -loadvm and -incoming too; but for those two we could make their monitor
commands exit preconfig mode automatically, and invoke them from the monitor
instead of specifying them on the command line.
Regarding: "what you want is effectively to execute monitor commands
from the migration stream"
That is not the goal of this series. It could be someone else's goal, when
fully developing a precreate phase, and in that context I understand and
agree with your comments. I have a narrower immediate problem to solve,
however.
For CPR, src qemu sends file descriptors to dst qemu using SCM_RIGHTS over
a dedicated channel, then src qemu sends migration state over the normal
migration channel.
Dst qemu reads the fds early, then calls the backend and device creation
functions which use them. Dst qemu then accepts and reads the migration
channel.
We need a way to send monitor commands that set dst migration capabilities,
before src qemu starts the migration. Hence the dst cannot proceed to
backend and device creation because the src has not sent fd's yet. Hence
we need a dst monitor before device creation. The precreate phase does that.
Regarding: "This series makes this much more complex."
I could simplify it if I abandon CPR for chardevs. Then
qemu_create_early_backends
and other early dependencies can remain as is. I would drop the notion of
a precreate phase, and instead leverage the preconfig phase. I would move
qemu_create_late_backends, and a small part at the end of qemu_init, to
qmp_x_exit_preconfig.
These patches from the series (slightly renamed) would suffice.
The "move preconfig boundary" patch is new.
* migration: init and listen during preconfig
* vl: move preconfig boundary
* monitor: connect in preconfig
* net: cleanup for preconfig phase
* migration: allow commands during preconfig
Would you consider supporting that?
- Steve
