Hi, In order to clean-up some hacks in qapi (having to unregister commands at runtime), I proposed a "[PATCH v5 02/20] qapi.py: add a simple #ifdef condition"
(see http://lists.gnu.org/archive/html/qemu-devel/2016-08/msg03106.html). However, we decided to drop that patch from the series and solve the problem later. The main issues were: - the syntax was awkward to the JSON schema and documentation - the evaluation of the condition was done in the qapi scripts, with very limited capability - each target/config would need different generated files. Instead, it could defer the #if evaluation to the C-preprocessor. With this series, top-level qapi JSON entity can take 'if' keys: { 'struct': 'TestIfStruct', 'data': { 'foo': 'int' }, 'if': 'defined(TEST_IF_STRUCT)' } Members can be exploded as dictionnary with 'type'/'if' keys: { 'struct': 'TestIfStruct', 'data': { 'foo': 'int', 'bar': { 'type': 'int', 'if': 'defined(TEST_IF_STRUCT_BAR)'} } } Enum values can be exploded as dictionnary with 'type'/'if' keys: { 'enum': 'TestIfEnum', 'data': [ 'foo', { 'name' : 'bar', 'if': 'defined(TEST_IF_ENUM_BAR)' } ] } A benefit from having conditional schema is that introspection will reflect more accurately the capability of the server. Another benefit is that it may help to remove some dead code when disabling a functionality. Starting from patch "qapi: add conditions to VNC type/commands/events on the schema", the series demonstrates adding conditions. The schema is splitted in common and per-target parts starting from "RFC: qapi: learn to split schema by 'top-unit'" patch. This allows a target schema to use target-poisoned defines as conditions (such as TARGET_I386). There are a lot more things one can make conditional in the QAPI schema, like pci/kvm/xen/numa/vde/slirp/posix/win32/vsock/lzo etc etc, however I am still evaluating the implication of such changes both externally and internally, for those interested, I can share my wip branch. Comments welcome, v5: - adapt to the new 'modular' / single generator design, making splitting schema a bit more complicated than before, when it was only at build-sys level. See "RFC: qapi: learn to split schema by 'top-unit'" for the new apporach at splitting common and target parts. - replace the #if-wrapper decorator with a @contextmanager (if necessary, this could be replaced by explicit begin/end calls instead now) - better split of target events, introducing seperate limiter/emitter - add a 'RFC: make RTC_CHANGE per-target' to show how conditional could be used and exercice the 'per-unit' event limiter - +painful rebase - add some r-b tags v4: - added "qlit: use QType instead of int" patch - use a "default:" case in qobject_from_qlit() - added a test for invalid 'if': [''] - listify ifcond in constructors - added "qapi: leave the ifcond attribute undefined until check()" patch (could be squashed) - use a negative lookahead in "qapi: mcgen() shouldn't indent # lines" - fix extra empty line in "qapi-introspect: add preprocessor conditions to generated QLit" - added desugar/normalization passes, such as NAME -> { 'name': NAME } - splitted "qapi: change enum visitor to take QAPISchemaMember" - prettify test-qapi.py members output - added missing #if wrapping for enum members in generated introspection - clarified some error messages - added "qapi: rename allow_dict to allow_implicit" patch for clarity - added a seperate patch for "qapi: add a dictionnary form" for enum names - added a seperate patch for "qapi: add a dictionnary form" for struct members - squashed the patches adding #if to generated code (some types generate both enum and struct members for example, so a step-by-step is unnecessarily complicated to deal with) - squashed some tests with related patch - added "qapi: add an error in case a discriminator is conditionnal" - add back VNC-specific crypto 'des-rfb' when VNC is disabled - change the way unit filtering is done (so no -u means no filtering) - made target.json the top-level documentation schema - removed some blank lines in generated code output - improve documentation, add various tests - commit message improvements - misc pycodestyle/pep8 fixes - add r-b tags v3: - rebased (qlit is now merged upstream) - solve the per-target #ifdef problem by using a target.json and new qapi generated target files - update some commit messages based on Markus review - more schema error reporting - move the ifcond argument closer to info/doc - use mcgen() in gen_if()/gen_endif() - simplify "modify to_qlit() to take an optional suffix" - fix generated qlit indentation - fix temporary build break by merging #if types & visitors patch - fix some redundant condtionals generation - change enum visitor to take QAPISchemaMember - reject unknown dictionnary keys in { .., 'if': ..} - split qapi test visitor print() with trailing ',' trick v2: after Markus review - "qboject: add literal qobject type", splitted & many updates - "qapi: generate a literal qobject for introspection", fixed leak - "visitor: pass size of strings array to enum visitor", rewrote as a series of patches to change char*[] to struct { int n, char *[] }. - split the "if" patches in parsing, tests, generation - added a docs/devel/qapi-code-gen.txt patch - added a patch to remove enum value assignment, to avoid "holes" with #if values - update #if VNC patch, compile out "info vnc", compile out des-rfb - dropped "qobject: replace dump_qobject() by qobject_to_string()" patch - dropped qapi.mk patch - other smaller changes Marc-André Lureau (49): qapi/visit: remove useless prefix argument qapi/events: generate event enum in main module qapi: add 'if' to top-level expressions qapi: pass 'if' condition into QAPISchemaEntity objects qapi: leave the ifcond attribute undefined until check() qapi: add 'ifcond' to visitor methods qapi: mcgen() shouldn't indent # lines qapi: add #if/#endif helpers qapi-introspect: modify to_qlit() to append ',' on level > 0 qapi-introspect: add preprocessor conditions to generated QLit qapi/commands: add #if conditions to commands qapi/events: add #if conditions to events qapi-types: refactor variants handling qapi-types: add #if conditions to types & visitors qapi: do not define enumeration value explicitely qapi: rename QAPISchemaEnumType.values to .members qapi: change enum visitor to take QAPISchemaMember tests: modify visit_enum_type() in test-qapi to print members qapi: factor out check_known_keys() qapi: add a dictionnary form with 'name' key for enum members qapi: add 'if' to enum members qapi-event: add 'if' condition to implicit event enum qapi: rename allow_dict to allow_implicit qapi: add a dictionary form with 'type' key for members qapi: add 'if' to implicit struct members qapi: add an error in case a discriminator is conditionnal qapi: add 'if' on union members qapi: add 'if' to alternate members qapi: add #if conditions to generated code members docs: document schema configuration qapi2texi: add 'If:' section to generated documentation qapi2texi: add 'If:' condition to enum values qapi2texi: add 'If:' condition to struct members qapi2texi: add condition to variants qapi: add conditions to VNC type/commands/events on the schema qapi: add conditions to SPICE type/commands/events on the schema qapi: add conditions to REPLICATION type/commands on the schema build-sys: move qmp-introspect per target qapi-commands: don't initialize command list in qmp_init_marshall() RFC: qapi: learn to split schema by 'top-unit' qapi: add a top-unit 'target' schema qapi: make rtc-reset-reinjection and SEV depend on TARGET_I386 qapi: make s390 commands depend on TARGET_S390X target.json: add a note about query-cpu* not being s390x-specific qapi: make query-gic-capabilities depend on TARGET_ARM qapi: make query-cpu-model-expansion depend on s390 or x86 qapi: make query-cpu-definitions depend on specific targets qapi: remove qmp_unregister_command() RFC: make RTC_CHANGE per-target qapi/block-core.json | 13 +- qapi/char.json | 10 +- qapi/migration.json | 12 +- qapi/misc.json | 380 ----------- qapi/qapi-schema.json | 1 + qapi/target.json | 415 ++++++++++++ qapi/ui.json | 75 ++- scripts/qapi/commands.py | 43 +- scripts/qapi/common.py | 599 +++++++++++++----- scripts/qapi/doc.py | 46 +- scripts/qapi/events.py | 45 +- scripts/qapi/introspect.py | 57 +- scripts/qapi/types.py | 98 +-- scripts/qapi/visit.py | 59 +- include/qapi/qmp-event.h | 4 + include/qapi/qmp/dispatch.h | 1 - include/sysemu/arch_init.h | 11 - target/i386/sev_i386.h | 2 +- ui/vnc.h | 2 + hmp.c | 9 +- hw/ppc/spapr_rtc.c | 2 +- hw/s390x/s390-skeys.c | 2 +- hw/timer/mc146818rtc.c | 4 +- migration/colo.c | 16 +- monitor.c | 297 +++++---- qapi/qmp-event.c | 11 + qapi/qmp-registry.c | 8 - qga/main.c | 1 + qmp.c | 72 +-- stubs/arch-query-cpu-def.c | 11 - stubs/arch-query-cpu-model-baseline.c | 13 - stubs/arch-query-cpu-model-comparison.c | 13 - stubs/arch-query-cpu-model-expansion.c | 13 - target/arm/helper.c | 3 +- target/arm/monitor.c | 2 +- target/i386/cpu.c | 6 +- target/ppc/translate_init.c | 3 +- target/s390x/cpu_models.c | 9 +- tests/test-qmp-cmds.c | 7 + tests/test-qobject-input-visitor.c | 1 - Makefile | 1 + Makefile.objs | 2 - Makefile.target | 4 + docs/devel/qapi-code-gen.txt | 38 ++ hmp-commands-info.hx | 2 + stubs/Makefile.objs | 4 - tests/Makefile.include | 14 +- tests/qapi-schema/alternate-base.err | 2 +- tests/qapi-schema/alternate-invalid-dict.err | 1 + ...ember.exit => alternate-invalid-dict.exit} | 0 tests/qapi-schema/alternate-invalid-dict.json | 4 + ...-member.out => alternate-invalid-dict.out} | 0 tests/qapi-schema/bad-if-empty-list.err | 1 + tests/qapi-schema/bad-if-empty-list.exit | 1 + tests/qapi-schema/bad-if-empty-list.json | 3 + tests/qapi-schema/bad-if-empty-list.out | 0 tests/qapi-schema/bad-if-empty.err | 1 + tests/qapi-schema/bad-if-empty.exit | 1 + tests/qapi-schema/bad-if-empty.json | 3 + tests/qapi-schema/bad-if-empty.out | 0 tests/qapi-schema/bad-if-list.err | 1 + tests/qapi-schema/bad-if-list.exit | 1 + tests/qapi-schema/bad-if-list.json | 3 + tests/qapi-schema/bad-if-list.out | 0 tests/qapi-schema/bad-if.err | 1 + tests/qapi-schema/bad-if.exit | 1 + tests/qapi-schema/bad-if.json | 3 + tests/qapi-schema/bad-if.out | 0 tests/qapi-schema/comments.out | 14 +- tests/qapi-schema/doc-bad-section.out | 13 +- tests/qapi-schema/doc-good.json | 11 +- tests/qapi-schema/doc-good.out | 23 +- tests/qapi-schema/doc-good.texi | 10 +- tests/qapi-schema/double-type.err | 2 +- tests/qapi-schema/empty.out | 9 +- tests/qapi-schema/enum-bad-member.err | 1 + tests/qapi-schema/enum-bad-member.exit | 1 + tests/qapi-schema/enum-bad-member.json | 2 + tests/qapi-schema/enum-bad-member.out | 0 .../qapi-schema/enum-dict-member-unknown.err | 1 + .../qapi-schema/enum-dict-member-unknown.exit | 1 + .../qapi-schema/enum-dict-member-unknown.json | 2 + .../qapi-schema/enum-dict-member-unknown.out | 0 tests/qapi-schema/enum-dict-member.err | 1 - tests/qapi-schema/enum-dict-member.json | 2 - tests/qapi-schema/enum-if-invalid.err | 1 + tests/qapi-schema/enum-if-invalid.exit | 1 + tests/qapi-schema/enum-if-invalid.json | 3 + tests/qapi-schema/enum-if-invalid.out | 0 tests/qapi-schema/enum-missing-data.err | 2 +- tests/qapi-schema/event-case.out | 9 +- tests/qapi-schema/event-nest-struct.err | 2 +- tests/qapi-schema/flat-union-inline.err | 2 +- .../flat-union-invalid-if-discriminator.err | 1 + .../flat-union-invalid-if-discriminator.exit | 1 + .../flat-union-invalid-if-discriminator.json | 17 + .../flat-union-invalid-if-discriminator.out | 0 tests/qapi-schema/ident-with-escape.out | 9 +- tests/qapi-schema/include-relpath.out | 14 +- tests/qapi-schema/include-repetition.out | 14 +- tests/qapi-schema/include-simple.out | 14 +- tests/qapi-schema/indented-expr.out | 9 +- tests/qapi-schema/nested-struct-data.err | 2 +- tests/qapi-schema/qapi-schema-test.json | 46 +- tests/qapi-schema/qapi-schema-test.out | 100 ++- .../struct-member-invalid-dict.err | 1 + .../struct-member-invalid-dict.exit | 1 + .../struct-member-invalid-dict.json | 3 + .../struct-member-invalid-dict.out | 0 tests/qapi-schema/test-qapi.py | 39 +- .../qapi-schema/union-branch-invalid-dict.err | 1 + .../union-branch-invalid-dict.exit | 1 + .../union-branch-invalid-dict.json | 4 + .../qapi-schema/union-branch-invalid-dict.out | 0 tests/qapi-schema/unknown-expr-key.err | 2 +- tests/qapi-schema/unknown-expr-key.json | 2 +- 116 files changed, 1768 insertions(+), 1094 deletions(-) create mode 100644 qapi/target.json mode change 100644 => 100755 scripts/qapi/doc.py delete mode 100644 stubs/arch-query-cpu-def.c delete mode 100644 stubs/arch-query-cpu-model-baseline.c delete mode 100644 stubs/arch-query-cpu-model-comparison.c delete mode 100644 stubs/arch-query-cpu-model-expansion.c create mode 100644 tests/qapi-schema/alternate-invalid-dict.err rename tests/qapi-schema/{enum-dict-member.exit => alternate-invalid-dict.exit} (100%) create mode 100644 tests/qapi-schema/alternate-invalid-dict.json rename tests/qapi-schema/{enum-dict-member.out => alternate-invalid-dict.out} (100%) create mode 100644 tests/qapi-schema/bad-if-empty-list.err create mode 100644 tests/qapi-schema/bad-if-empty-list.exit create mode 100644 tests/qapi-schema/bad-if-empty-list.json create mode 100644 tests/qapi-schema/bad-if-empty-list.out create mode 100644 tests/qapi-schema/bad-if-empty.err create mode 100644 tests/qapi-schema/bad-if-empty.exit create mode 100644 tests/qapi-schema/bad-if-empty.json create mode 100644 tests/qapi-schema/bad-if-empty.out create mode 100644 tests/qapi-schema/bad-if-list.err create mode 100644 tests/qapi-schema/bad-if-list.exit create mode 100644 tests/qapi-schema/bad-if-list.json create mode 100644 tests/qapi-schema/bad-if-list.out create mode 100644 tests/qapi-schema/bad-if.err create mode 100644 tests/qapi-schema/bad-if.exit create mode 100644 tests/qapi-schema/bad-if.json create mode 100644 tests/qapi-schema/bad-if.out create mode 100644 tests/qapi-schema/enum-bad-member.err create mode 100644 tests/qapi-schema/enum-bad-member.exit create mode 100644 tests/qapi-schema/enum-bad-member.json create mode 100644 tests/qapi-schema/enum-bad-member.out create mode 100644 tests/qapi-schema/enum-dict-member-unknown.err create mode 100644 tests/qapi-schema/enum-dict-member-unknown.exit create mode 100644 tests/qapi-schema/enum-dict-member-unknown.json create mode 100644 tests/qapi-schema/enum-dict-member-unknown.out delete mode 100644 tests/qapi-schema/enum-dict-member.err delete mode 100644 tests/qapi-schema/enum-dict-member.json create mode 100644 tests/qapi-schema/enum-if-invalid.err create mode 100644 tests/qapi-schema/enum-if-invalid.exit create mode 100644 tests/qapi-schema/enum-if-invalid.json create mode 100644 tests/qapi-schema/enum-if-invalid.out create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.err create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.exit create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.json create mode 100644 tests/qapi-schema/flat-union-invalid-if-discriminator.out create mode 100644 tests/qapi-schema/struct-member-invalid-dict.err create mode 100644 tests/qapi-schema/struct-member-invalid-dict.exit create mode 100644 tests/qapi-schema/struct-member-invalid-dict.json create mode 100644 tests/qapi-schema/struct-member-invalid-dict.out create mode 100644 tests/qapi-schema/union-branch-invalid-dict.err create mode 100644 tests/qapi-schema/union-branch-invalid-dict.exit create mode 100644 tests/qapi-schema/union-branch-invalid-dict.json create mode 100644 tests/qapi-schema/union-branch-invalid-dict.out -- 2.16.2.521.g9aa15f885a