v5->v6:
* add exit-preconfig QMP command instead of overloading meaning of 'cont'
command
* add doc text to qemu-tech.texi about -S and --preconfig
* add numa configuration example into commit message of 10/11
* limit set-numa-node QMP command to preconfig mode
v4->v5:
* rebase on top of OOB changes that's now in master
* s/qobject_to_qdict(/qobject_to(QDict,/
* s/-preconfig/--preconfig/
* s/2.12/2.13/
* s/parse_NumaOptions/set_numa_options/
* drop if (err) guard around error_propagate()
* move QCO_ALLOWED_IN_PRECONFIG and do_qmp_dispatch() runstate check
from the later patch 'qapi: introduce new cmd option
"allowed-in-preconfig"'
to here for better bissectability
* add TODO comment to '{ RUN_STATE_PRECONFIG, RUN_STATE_INMIGRATE }'
transition
* check for incoming && preconfig outside of option parsing loop
* add 'use QMP instead" to error message, suggesting user the right interface
to use
* allow query-command-line-options in preconfig state
* make sure that allowed-in-preconfig could be set only to True
* move out QCO_ALLOWED_IN_PRECONFIG check in do_qmp_dispatch() to
earlier 'cli: add --preconfig option' patch
* 2 extra test patches
'tests: let qapi-schema tests detect allowed-in-preconfig'
'tests: add allowed-in-preconfig-test for qapi-schema'
v3->v4:
* replace 'runstates' list in QMP command with a single
boolean 'ption allowed-in-preconfig' like it's done with
'allow-oob'. Which allows to simplify intrusive QAPI
changes quite a lot. (Eric Blake <ebl...@redhat.com>)
* Make sure HMP is disbled for real, v3 was just printing
error mesage but allowing command to be executed
("Dr. David Alan Gilbert" <dgilb...@redhat.com>)
* Improve "cli: add -preconfig option" commit message,
explain a bit more on semantics of new state/option.
* ithe rest of minor fixups suggested at v3 review
(Eric Blake <ebl...@redhat.com>)
PS:
havn't impl. test for new option in
tests/qapi-schema/qapi-schema-test.json yet,
can do it on top if approach is acceptable.
v1->v3:
* introduce PRECONFIG runstate with -preconfig option.
it's cleaner to manage transitions and do checks
than reusing existing PRELAUNCH state.
* extend QAPI schema commands with 'runstates' keyword,
so that it would be possible to specify in command definition
when it is valid to execute.
(python changes a bit hackery, since I have little to
no idea how it should work)
* add preconfig QMP and set-numa-node tests
* make mutually exclusive -preconfig and -incoming options,
for simplicity sake. Shouldn't be problem as target can
be starter with pure CLI, since mapping on source is
already known.
* Drop HMP part and leave only QMP in preconfig state.
Currently it's problematic to configure NUMA mapping for CPUs using "-numa cpu="
option, without restarting QEMU, first is to query board specific CPU layout
with query-hotpluggable-cpus QMP command (which is function of -M and -smp CLI
options). Mgmt side isn't happy with restarting QEMU to query configuration
parameters first and then run actual instance, so it keeps using old cpu_index
based option '-numa cpus' instead of new socket/core/thread-id based one.
Introduce a new '--preconfig' CLI option which allows pausing QEMU before
machine_init() is run (preconfig state). So it would be possible to query
possible CPUs layout, configure NUMA mapping based on query result with
set-numa-node QMP command and continue executing without restarting QEMU.
Difference between new --preconfig pause point vs -S is that the later pauses
QEMU after machine is constructed and ready to run guest code or in process of
incoming migration (essentially machine is in some running state (with paused
VCPUs) and any action on it is considered as hotplug). At this point it's hard
to configure or reconfigure parameters that affect machine_init() and later
stages. While the new --preconfig option pauses QEMU instance before
machine_init() and would allow to configure parameters as if doing it from CLI
but in interactve manner using QMP interface, which would allow introspecting
and configuring QEMU instance without restarting it.
Initially only limited set of commands (that are ready to work with non
initialized machine or without it) are allowed in preconfig state:
#existing commands:
qmp_capabilities
query-qmp-schema
query-commands
query-command-line-options
query-status
query-hotpluggable-cpus
#new commands
exit-preconfig
set-numa-node
Which commands are allowed at preconfig state, is defined by QAPI Schema
with help of optional 'allowed-in-preconfig' flag in command definition,
which allows users to get list of commands it can run with help of
query-qmp-schema command.
Once user done with configuration at preconfig state, it could be exited with
exit-preconfig command, in which case QEMU would proceed with VM intialization
and either start guest execution or pause in prelaunch state if -S CLI option
were specified along with --prelaunch on startup.
PS:
Later we can modify other commands to run early, for example device_add. So
that devices would be added in cold-plug context, instead of getting device
'hotplugged' with followed up fixups and workarounds to make it look like
cold-plugged.
Example of configuration CPU's NUMA mapping with --preconfig:
$QEMU -smp 2 --preconfig ...
QMP:
# get CPUs layout for current target/machine/CLI
-> {'execute': 'query-hotpluggable-cpus' }
<- {'return': [
{'props': {'core-id': 0, 'thread-id': 0, 'socket-id': 1}, ... },
{'props': {'core-id': 0, 'thread-id': 0, 'socket-id': 0}, ... }
]}
# configure 1st node
-> {'execute': 'set-numa-node', 'arguments': { 'type': 'node', 'nodeid': 0 } }
<- {'return': {}}
-> {'execute': 'set-numa-node', 'arguments': { 'type': 'cpu',
'node-id': 0, 'core-id': 0, 'thread-id': 0, 'socket-id': 1, }
}
<- {'return': {}}
# configure 2nd node
-> {'execute': 'set-numa-node', 'arguments': { 'type': 'node', 'nodeid': 1 } }
-> {'execute': 'set-numa-node', 'arguments': { 'type': 'cpu',
'node-id': 1, 'core-id': 0, 'thread-id': 0, 'socket-id': 0 }
}
<- {'return': {}}
# [optional] verify configuration
-> {'execute': 'query-hotpluggable-cpus' }
<- {'return': [
{'props': {'core-id': 0, 'thread-id': 0, 'node-id': 0, 'socket-id': 1},
... },
{'props': {'core-id': 0, 'thread-id': 0, 'node-id': 1, 'socket-id': 0},
... }
]}
Git tree:
https://github.com/imammedo/qemu.git qmp_preconfig_v6
CC: ebl...@redhat.com
CC: ehabk...@redhat.com
CC: pkre...@redhat.com
CC: arm...@redhat.com
Igor Mammedov (11):
numa: postpone options post-processing till machine_run_board_init()
numa: split out NumaOptions parsing into set_numa_options()
cli: add --preconfig option
hmp: disable monitor in preconfig state
qapi: introduce new cmd option "allowed-in-preconfig"
tests: let qapi-schema tests detect allowed-in-preconfig
tests: add allowed-in-preconfig-test for qapi-schema
tests: extend qmp test with preconfig checks
qmp: permit query-hotpluggable-cpus in preconfig state
qmp: add set-numa-node command
tests: functional tests for QMP command set-numa-node
include/qapi/qmp/dispatch.h | 1 +
include/sysemu/numa.h | 2 +
include/sysemu/sysemu.h | 1 +
tests/libqtest.h | 9 +++
docs/devel/qapi-code-gen.txt | 10 +++-
hw/core/machine.c | 5 +-
monitor.c | 11 +++-
numa.c | 70 ++++++++++++++++--------
qapi/introspect.json | 5 +-
qapi/misc.json | 48 ++++++++++++++--
qapi/qmp-dispatch.c | 8 +++
qapi/run-state.json | 8 ++-
qemu-options.hx | 13 +++++
qemu-tech.texi | 40 ++++++++++++++
qmp.c | 10 ++++
scripts/qapi/commands.py | 12 ++--
scripts/qapi/common.py | 19 ++++---
scripts/qapi/doc.py | 4 +-
scripts/qapi/introspect.py | 7 ++-
tests/Makefile.include | 1 +
tests/libqtest.c | 7 +++
tests/numa-test.c | 61 +++++++++++++++++++++
tests/qapi-schema/allowed-in-preconfig-test.err | 1 +
tests/qapi-schema/allowed-in-preconfig-test.exit | 1 +
tests/qapi-schema/allowed-in-preconfig-test.json | 2 +
tests/qapi-schema/allowed-in-preconfig-test.out | 0
tests/qapi-schema/doc-good.out | 4 +-
tests/qapi-schema/ident-with-escape.out | 2 +-
tests/qapi-schema/indented-expr.out | 4 +-
tests/qapi-schema/qapi-schema-test.json | 3 +
tests/qapi-schema/qapi-schema-test.out | 22 ++++----
tests/qapi-schema/test-qapi.py | 8 +--
tests/qmp-test.c | 37 +++++++++++++
tests/test-qmp-cmds.c | 4 ++
vl.c | 35 +++++++++++-
35 files changed, 403 insertions(+), 72 deletions(-)
create mode 100644 tests/qapi-schema/allowed-in-preconfig-test.err
create mode 100644 tests/qapi-schema/allowed-in-preconfig-test.exit
create mode 100644 tests/qapi-schema/allowed-in-preconfig-test.json
create mode 100644 tests/qapi-schema/allowed-in-preconfig-test.out
--
2.7.4
