Hi,
no logical changes this time.
In this version, cross-compilation and compilation with mingw32 are fixed.
Our feature generation script gen-features has to be compiled for the
host architecture, where we do the compilation and bitmaps cannot be
directly initialized from the generated data, because they are based on
unsigned long.
So now, we compile wih HOST_CC and made gen-features independant from QEMU
includes, and we initialize the bitmaps from generated data during startup.
Latest version can be found on branch:
github.com/cohuck/qemu cpumodel-s390x-v4
--------------------------------Concept------------------------------------
This is our second attempt to implement CPU models for s390x. We realized
that we also want to have features exposed via the CPU model. While doing
that we realized that we want to have a better interface for libvirt.
Unfortunately, CPU models on s390x are special and we have to take care of:
- A CPU like z13 looks differently in various environments (under different
LPAR versions, under different z/VM versions, under different KVM
versions, export regulation) - we have _a lot_ of feature variability.
- We still have certain features that are not published but might be
implemented/introduced in the future. As they are a theoretical part
of a CPU already, we have to find a way to model these future changes.
- We still have certain features that are already published, but not
implemented. Implementation might be added in the future in KVM.
- We heavily rely on KVM to tell us which features it can actually
virtualize - user space queries like "STFL(e)" give no guarantees.
- Certain "subfeatures" were introduced in one run. In practice, they are
always around, but in theory, subfeatures could be dropped in the future.
- Just because two CPU models have the same features doesn't mean they
are equal - some internal numbers might be different. E.g. we won't allow
running a z13 under a zBC12 just by turning off features.
- We cannot blindly enable all possible features for a CPU generation,
the IBC "Instruction Blocking Control" in KVM will try to block
instructions introduced with certain features. So a CPU generation always
has some maximum feature set that is guaranteed to work.
It all boils down to a specific released CPU to have.
a) A minimum feature set that we expect it to be have on every hypervisor.
b) A variable part that depends on the hypervisor and that could be
extended in the future (adding not yet implemented features) that we
always want to enable later on.
c) A variable part that we want to enable only if requested - nested
virtualization ("vsie") and assists are one example.
As we want our users to always make use of most features, e.g. when using
the "z13" CPU model, we will have to update our CPU models between QEMU
releases, to include the latest feature additions we implemented/unlocked.
We're badically trying to be able at one point in the future to really look
like a maximum "z13" CPU in QEMU. However, that means that a "z13" CPU
looks differently on different QEMU versions. We will make sure using
compatibility machines, that migration stays safe.
However, if the feature set of a CPU model is bound to a compatibility
machine, how can it be of any help when used without a compatibility
machine? E.g. when indicating the host CPU model in "virsh capabilities",
simply saying "z13" is not really clear. Also when baselining and
comparing CPU models, we usually have no compatibility machine "context"
at hand. For this reason we introduce "static" CPU models, which will
never change, so everybody knows without a compatibility machine, what we
are talking about.
CPU definitions/models can be categorized:
1. migratable: all features _can_ be identified + properly migrated.
--> With migratable=off on x86, "host" cannot be migrated properly.
2. migration-safe: in combination with a QEMU machine, migration will work.
--> No CPU features are lost/added during migration
3. static: not bound to a QEMU machine - featureset will never* change.
--> Everybody knows which features are part of it, not affected by a compat
machine. "virsh capabilities" can show this as "host" model.
*except for trivial bugfixes, especially on very old models nobody used in
production. Should be avoided.
We are currently planning to also support a "migratable=off" on s390x
for the "-cpu host" model, in order to enable all features, including not
recognized ones.
So we have on s390x:
a) A static CPU model for each released CPU that will never change and
maps to the minimum feature set we expect to be around on all
hypervisors. e.g. "z13-base" or "z10EC.2-base".
b) A "default" CPU model for each released CPU, that can change between
QEMU versions and that will always include the features we expect to
be around in our currently supported environments and will contain only
features we expect to be stable. E.g. nested virtualization will not be
contained in these models for now. "z13" or "z10EC.2" are examples.
c) An internal "maximum" CPU model for each generation that tells us which
features were supported as a maximum back when the hardware was
released. This will not be exposed but used internally.
To not have to replicate all CPU model changes ("new default fetaures") in
libvirt, to not duplicate the logic about compatibility and the like,
our approach tries to keep all the QEMU logic in libvirt and provide
standardized interfaces for libvirt to e.g. baseline, compare. This
allows libvirt to not have to care about any model names or feature names,
it can just pass the data from interface to interface and report it to
the user.
Also, libvirt might want to know what the "host" model looks like and
convert a CPU model to a migration safe variant. For this reason, a QMP
command is added that can create a migration safe variant of a variable
CPU model, indicating only the delta changes done to a stable model.
So we have:
a) "query-cpu-model-expansion" - tell us what the "host" or another CPU
model looks like. Either falling back to a static model or
completely exposing all properties.
b) "query-cpu-model-comparison" - tell us how two CPU models compare,
-> needed for "virsh cpu-compare"
c) "query-cpu-model-baseline" - create a new model out of two models,
-> needed for virsh cpu-baseline"
--------------------------------Libvirt usecase----------------------------
Testing for runability:
- Simply try to start QEMU with KVM, compat machine, CPU model
- Could be done using query-cpu-model-comparison in the future.
Identifying host model, e.g. "virsh capabilities"
- query-cpu-model-expansion on "host" with "-M none --enable-kvm"
<cpu mode='host-model'>:
- simply copy the identified host model
<cpu mode='host-passthrough'>:
- "-cpu host"
"virsh cpu-baseline":
- query-cpu-model-baseline on two models with "-M none"
"virsh cpu-compare":
- query-cpu-model-comparison on two models with "-M none"
There might be some cenarios where libvirt wants to convert another CPU
model to a static variant, this can be done using query-cpu-model-expansion
--------------------------------QMP examples------------------------------
Expanding "host" on z13.2 without nested virtualization support:
{ "execute":"query-cpu-model-expansion",
"arguments":{
"type":"static",
"model":{
"name":"host"
}}}
{ "return":{
"model":{
"name":"z13.2-base",
"props":{
"aefsi":true,
"msa5":true,
"msa4":true,
"msa3":true,
"msa2":true,
"msa1":true,
"sthyi":true,
"edat":true,
"ri":true,
"edat2":true,
"vx":true,
"ipter":true,
"esop":true,
"cte":true,
"te":true,
"cmm":true
}}}}
Expanding "host" on zEC12.2 without nested virtualization support:
{ "execute":"query-cpu-model-expansion",
"arguments":{
"type":"static",
"model":{
"name":"host"
}}}
{ "return":{
"model":{
"name":"zEC12.2-base",
"props":{
"aefsi":true,
"msa5":true,
"msa4":true,
"msa3":true,
"msa2":true,
"msa1":true,
"sthyi":true,
"edat":true,
"ri":true,
"edat2":true,
"ipter":true,
"cei":true,
"gpereh":true,
"esop":true,
"ib":true,
"siif":true,
"ibs":true,
"sief2":true,
"cte":true,
"64bscao":true,
"te":true
"cmm":true,
"gsls":true,
}}}}
Comparing both models (dropping features for readability)
{ "execute":"query-cpu-model-comparison",
"arguments":{
"modela":{
"name":"z13.2-base",
"props":{ ... }
},
"modelb":{
"name":"zEC12.2-base",
"props":{ ... }
}
}}}
{"return":{
"result":"incompatible",
"responsible-properties":[
"cei",
"ib",
"siif",
"gpereh",
"sief2",
"ibs",
"64bscao",
"gsls",
"vx",
"dfppc",
"gen13e",
"gen13ptff",
"type"
]
}}
Indicated are all z13 features + nested virtualization features. "type"
indicates that both models can never be made identical (as different
CPU generations).
Baselining both models (dropping features for readability)
{ "execute":"query-cpu-model-baseline",
"arguments":{
"modela":{
"name":"z13.2-base",
"props":{ ... }
},
"modelb":{
"name":"zEC12.2-base",
"props":{ ... }
}}}}
{
"return":{
"model":{
"name":"zEC12.2-base",
"props":{
"aefsi":true,
"msa5":true,
"msa4":true,
"msa3":true,
"msa2":true,
"msa1":true,
"sthyi":true,
"edat":true,
"ri":true,
"edat2":true,
"ipter":true,
"esop":true,
"cte":true,
"te":true
"cmm":true,
}}}}
Which is in fact a zEC12.2 without nested virtualization.
As an example, what "host" on z/VM system (zEC12.2, no vSIE) expands to:
{ "execute":"query-cpu-model-expansion",
"arguments":{
"type":"static",
"model":{
"name":"host"
}}}
{ "return":{
"model":{
"name":"zEC12.2-base",
"props":{
"aefsi":true,
"msa5":true,
"msa4":true,
"msa3":true,
"msa2":true,
"msa1":true,
"sthyi":true,
"esop":true,
"cte":true,
}}}}
Important is: no TE, no EDAT(2), no CMM, no RI, no IPTER
--------------------------------Changes-----------------------------------
Patch v3 -> v4:
- Split out CPU feature numbers into cpu_features_def.h
- gen-features is now compiled via HOST_CC and doesn't require any QEMU
includes any more except cpu_features_def.h. CC FLAGS are dropped.
gen-features is now completely standalone.
- Feature bitmaps (for groups and CPU models) are initialized on startup
(via module_init) from the generated data, as bitmaps cannot be properly
initialized from static data (because they are unsigned long arrays)
Patch v2 -> v3:
- Two fixes to make checkpatch (more) happy. Ageed to ignore the remaining
reports.
- Renamed a couple of features as requested by Christian.
- Added "s390x/kvm: don't enable key wrapping if msa3 is disabled" to
avoid errors when running CPU models without MSA3.
Patch v1 -> v2:
- Removed a leftover return statement in s390_get_feat_block()
- Replaced additional vector CAP checks by s390_has_feat()
- Report an error on unsupported expansion types in s390x
- Adapted documentation of query-cpu-definitions return values
-- static is not optional anymore, it is automatically set to false
- Changed documentation of query-cpu-(expansion|comparions|baseline)
-- be more verbose, document more clearly what can affect the result
-- added more details on how the interfaces can be used
-- adapted all suggestions by Eduardo
- new kvm/next linux-headers sync
- s390x:
-- Renamed feature "tx" to "te" (match /proc/cpuinfo)
-- Renamed feature "ctx" to "cte" (match te)
-- Renamed feature "z" to "zarch" (match /proc/cpuinfo)
-- Renamed feature "ecput" to "ectg" (instruction name)
-- Renamed feature "exece" to "exrl" (instruction name)
-- Renamed feature "spropg" to "sprogp" (misspelled)
-- Renamed feature "dateh1" to "dateh" (facility 1 is implicit)
-- Renamed feature "edat1" to "edat" (facility 1 is implicit)
-- Removed dependency check "cte" -> "te" to avoid warnings under z/VM
-- Added sthyi to every default + max model having stfle
-- Removed store-CPU-counter-multiple facility for now
RFC -> Patch v1:
- Rebased to master, had to fixup qmp_visitor and CPU ceation stuff
- Changed function calls from (*fn)(...) to fn(...)
- Mark models not only as migration-safe but also as static
- Introduced "qmp: details about CPU definitions in query-cpu-definitions"
- Expose "static" as CPU class property
- All CPU models except "host" are now migration-safe (e.g. qemu and z13)
- Replaced linux-header update by a proper kvm/next update
- Removed baseline type parameter from "query-cpu-model-baseline" to KIS
- Renamed "stable" expansion to "static" (as stable sounds misleading)
- Removed "migratable" expansion type, can be done via "migratable=off"
- Heavily improved description of the new QMP interfaces
- Actually tested all QMP interfaces. Various bugfixes.
David Hildenbrand (28):
qmp: details about CPU definitions in query-cpu-definitions
s390x/cpumodel: "host" and "qemu" as CPU subclasses
s390x/cpumodel: expose CPU class properties
s390x/cpumodel: generate CPU feature group lists
s390x/cpumodel: introduce CPU feature group definitions
s390x/cpumodel: register defined CPU models as subclasses
s390x/cpumodel: store the CPU model in the CPU instance
s390x/cpumodel: expose features and feature groups as properties
s390x/cpumodel: let the CPU model handle feature checks
s390x/cpumodel: check and apply the CPU model
s390x/sclp: factor out preparation of cpu entries
s390x/sclp: introduce sclp feature blocks
s390x/sclp: indicate sclp features
s390x/sclp: propagate the ibc val(lowest and unblocked ibc)
s390x/sclp: propagate the mha via sclp
s390x/sclp: propagate hmfai
linux-headers: update against kvm/next
s390x/kvm: allow runtime-instrumentation for "none" machine
s390x/kvm: implement CPU model support
s390x/kvm: disable host model for problematic compat machines
s390x/kvm: let the CPU model control CMM(A)
s390x/kvm: don't enable key wrapping if msa3 is disabled
qmp: add QMP interface "query-cpu-model-expansion"
qmp: add QMP interface "query-cpu-model-comparison"
qmp: add QMP interface "query-cpu-model-baseline"
s390x/cpumodel: implement QMP interface "query-cpu-model-expansion"
s390x/cpumodel: implement QMP interface "query-cpu-model-comparison"
s390x/cpumodel: implement QMP interface "query-cpu-model-baseline"
Michael Mueller (2):
s390x/cpumodel: introduce CPU features
s390x/cpumodel: generate CPU feature lists for CPU models
Makefile.target | 2 +-
hw/s390x/s390-virtio-ccw.c | 5 +
hw/s390x/s390-virtio.c | 6 +-
hw/s390x/sclp.c | 35 +-
include/hw/s390x/sclp.h | 17 +-
include/standard-headers/linux/input-event-codes.h | 32 +
include/standard-headers/linux/input.h | 1 +
include/standard-headers/linux/virtio_config.h | 10 +-
include/standard-headers/linux/virtio_ids.h | 1 +
include/standard-headers/linux/virtio_net.h | 3 +
include/sysemu/arch_init.h | 9 +
linux-headers/asm-arm/kvm.h | 4 +-
linux-headers/asm-arm64/kvm.h | 2 +
linux-headers/asm-s390/kvm.h | 41 +
linux-headers/asm-x86/unistd_x32.h | 4 +-
linux-headers/linux/kvm.h | 18 +-
linux-headers/linux/vhost.h | 33 +
qapi-schema.json | 246 ++++-
qmp-commands.hx | 18 +
qmp.c | 21 +
rules.mak | 1 +
stubs/Makefile.objs | 3 +
stubs/arch-query-cpu-model-baseline.c | 12 +
stubs/arch-query-cpu-model-comparison.c | 12 +
stubs/arch-query-cpu-model-expansion.c | 12 +
target-s390x/Makefile.objs | 22 +-
target-s390x/cpu-qom.h | 6 +
target-s390x/cpu.c | 35 +-
target-s390x/cpu.h | 5 +
target-s390x/cpu_features.c | 404 +++++++
target-s390x/cpu_features.h | 93 ++
target-s390x/cpu_features_def.h | 231 ++++
target-s390x/cpu_models.c | 1100 ++++++++++++++++++++
target-s390x/cpu_models.h | 119 +++
target-s390x/gen-features.c | 592 +++++++++++
target-s390x/helper.c | 33 +-
target-s390x/kvm.c | 356 ++++++-
target-s390x/machine.c | 14 +-
38 files changed, 3480 insertions(+), 78 deletions(-)
create mode 100644 stubs/arch-query-cpu-model-baseline.c
create mode 100644 stubs/arch-query-cpu-model-comparison.c
create mode 100644 stubs/arch-query-cpu-model-expansion.c
create mode 100644 target-s390x/cpu_features.c
create mode 100644 target-s390x/cpu_features.h
create mode 100644 target-s390x/cpu_features_def.h
create mode 100644 target-s390x/cpu_models.c
create mode 100644 target-s390x/cpu_models.h
create mode 100644 target-s390x/gen-features.c
--
2.8.4
