On Wed, Jun 12, 2019 at 06:04:25PM +0200, Cédric Le Goater wrote: > This includes various small updates and a better description of the > chosen interrupt mode resulting from the combination of the 'ic-mode' > machine option, the 'kernel_irqchip' option, guest support and KVM > support. > > Signed-off-by: Cédric Le Goater <c...@kaod.org> > --- > > David, please check the tables below. Thanks
LGTM, applied.
>
> docs/specs/ppc-spapr-xive.rst | 112 ++++++++++++++++++++++++++++++++--
> docs/specs/ppc-xive.rst | 9 +--
> 2 files changed, 111 insertions(+), 10 deletions(-)
>
> diff --git a/docs/specs/ppc-spapr-xive.rst b/docs/specs/ppc-spapr-xive.rst
> index 539ce7ca4e90..7a64c9d04951 100644
> --- a/docs/specs/ppc-spapr-xive.rst
> +++ b/docs/specs/ppc-spapr-xive.rst
> @@ -34,19 +34,118 @@ CAS Negotiation
> ---------------
>
> QEMU advertises the supported interrupt modes in the device tree
> -property "ibm,arch-vec-5-platform-support" in byte 23 and the OS
> -Selection for XIVE is indicated in the "ibm,architecture-vec-5"
> +property ``ibm,arch-vec-5-platform-support`` in byte 23 and the OS
> +Selection for XIVE is indicated in the ``ibm,architecture-vec-5``
> property byte 23.
>
> The interrupt modes supported by the machine depend on the CPU type
> (POWER9 is required for XIVE) but also on the machine property
> ``ic-mode`` which can be set on the command line. It can take the
> -following values: ``xics``, ``xive``, ``dual`` and currently ``xics``
> -is the default but it may change in the future.
> +following values: ``xics``, ``xive``, and ``dual`` which is the
> +default mode. ``dual`` means that both modes XICS **and** XIVE are
> +supported and if the guest OS supports XIVE, this mode will be
> +selected.
>
> The choosen interrupt mode is activated after a reconfiguration done
> in a machine reset.
>
> +KVM negotiation
> +---------------
> +
> +When the guest starts under KVM, the capabilities of the host kernel
> +and QEMU are also negotiated. Depending on the version of the host
> +kernel, KVM will advertise the XIVE capability to QEMU or not.
> +
> +Nevertheless, the available interrupt modes in the machine should not
> +depend on the XIVE KVM capability of the host. On older kernels
> +without XIVE KVM support, QEMU will use the emulated XIVE device as a
> +fallback and on newer kernels (>=5.2), the KVM XIVE device.
> +
> +As a final refinement, the user can also switch the use of the KVM
> +device with the machine option ``kernel_irqchip``.
> +
> +
> +XIVE support in KVM
> +~~~~~~~~~~~~~~~~~~~
> +
> +For guest OSes supporting XIVE, the resulting interrupt modes on host
> +kernels with XIVE KVM support are the following:
> +
> +============== ============= ============= ================
> +ic-mode kernel_irqchip
> +-------------- ----------------------------------------------
> +/ allowed off on
> + (default)
> +============== ============= ============= ================
> +dual (default) XIVE KVM XIVE emul. XIVE KVM
> +xive XIVE KVM XIVE emul. XIVE KVM
> +xics XICS KVM XICS emul. XICS KVM
> +============== ============= ============= ================
> +
> +For legacy guest OSes without XIVE support, the resulting interrupt
> +modes are the following:
> +
> +============== ============= ============= ================
> +ic-mode kernel_irqchip
> +-------------- ----------------------------------------------
> +/ allowed off on
> + (default)
> +============== ============= ============= ================
> +dual (default) XICS KVM XICS emul. XICS KVM
> +xive QEMU error(3) QEMU error(3) QEMU error(3)
> +xics XICS KVM XICS emul. XICS KVM
> +============== ============= ============= ================
> +
> +(3) QEMU fails at CAS with ``Guest requested unavailable interrupt
> + mode (XICS), either don't set the ic-mode machine property or try
> + ic-mode=xics or ic-mode=dual``
> +
> +
> +No XIVE support in KVM
> +~~~~~~~~~~~~~~~~~~~~~~
> +
> +For guest OSes supporting XIVE, the resulting interrupt modes on host
> +kernels without XIVE KVM support are the following:
> +
> +============== ============= ============= ================
> +ic-mode kernel_irqchip
> +-------------- ----------------------------------------------
> +/ allowed off on
> + (default)
> +============== ============= ============= ================
> +dual (default) XIVE emul.(1) XIVE emul. QEMU error (2)
> +xive XIVE emul.(1) XIVE emul. QEMU error (2)
> +xics XICS KVM XICS emul. XICS KVM
> +============== ============= ============= ================
> +
> +
> +(1) QEMU warns with ``warning: kernel_irqchip requested but unavailable:
> + IRQ_XIVE capability must be present for KVM``
> +(2) QEMU fails with ``kernel_irqchip requested but unavailable:
> + IRQ_XIVE capability must be present for KVM``
> +
> +
> +For legacy guest OSes without XIVE support, the resulting interrupt
> +modes are the following:
> +
> +============== ============= ============= ================
> +ic-mode kernel_irqchip
> +-------------- ----------------------------------------------
> +/ allowed off on
> + (default)
> +============== ============= ============= ================
> +dual (default) QEMU error(4) XICS emul. QEMU error(4)
> +xive QEMU error(3) QEMU error(3) QEMU error(3)
> +xics XICS KVM XICS emul. XICS KVM
> +============== ============= ============= ================
> +
> +(3) QEMU fails at CAS with ``Guest requested unavailable interrupt
> + mode (XICS), either don't set the ic-mode machine property or try
> + ic-mode=xics or ic-mode=dual``
> +(4) QEMU/KVM incompatibility due to device destruction in reset. This
> + needs to be addressed more cleanly with an error.
> +
> +
> XIVE Device tree properties
> ---------------------------
>
> @@ -92,10 +191,11 @@ for both interrupt mode. The different ranges are
> defined as follow :
> - ``0x0000 .. 0x0FFF`` 4K CPU IPIs (only used under XIVE)
> - ``0x1000 .. 0x1000`` 1 EPOW
> - ``0x1001 .. 0x1001`` 1 HOTPLUG
> +- ``0x1002 .. 0x10FF`` unused
> - ``0x1100 .. 0x11FF`` 256 VIO devices
> -- ``0x1200 .. 0x127F`` 32 PHBs devices
> +- ``0x1200 .. 0x127F`` 32x4 LSIs for PHB devices
> - ``0x1280 .. 0x12FF`` unused
> -- ``0x1300 .. 0x1FFF`` PHB MSIs
> +- ``0x1300 .. 0x1FFF`` PHB MSIs (dynamically allocated)
>
> Monitoring XIVE
> ---------------
> diff --git a/docs/specs/ppc-xive.rst b/docs/specs/ppc-xive.rst
> index b997dc062910..148d57eb6ab2 100644
> --- a/docs/specs/ppc-xive.rst
> +++ b/docs/specs/ppc-xive.rst
> @@ -20,10 +20,11 @@ The XIVE IC is composed of three sub-engines, each taking
> care of a
> processing layer of external interrupts:
>
> - Interrupt Virtualization Source Engine (IVSE), or Source Controller
> - (SC). These are found in PCI PHBs, in the PSI host bridge
> - controller, but also inside the main controller for the core IPIs
> - and other sub-chips (NX, CAP, NPU) of the chip/processor. They are
> - configured to feed the IVRE with events.
> + (SC). These are found in PCI PHBs, in the Processor Service
> + Interface (PSI) host bridge Controller, but also inside the main
> + controller for the core IPIs and other sub-chips (NX, CAP, NPU) of
> + the chip/processor. They are configured to feed the IVRE with
> + events.
> - Interrupt Virtualization Routing Engine (IVRE) or Virtualization
> Controller (VC). It handles event coalescing and perform interrupt
> routing by matching an event source number with an Event
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
signature.asc
Description: PGP signature
