Hello,
This series adds support for MSI IRQ domains on top of the XICS (P8)
and XIVE (P9/P10) IRQ domains for the PowerNV (baremetal) and pSeries
(VM) platforms. It should simplify and improve IRQ affinity of PCI
MSIs under these PowerPC platforms, specially for drivers distributing
multiple RX/TX queues on the different CPUs of the system.
Data locality can still be improved with an interrupt controller node
per chip but this requires FW changes. It could be done under OPAL.
The patchset has a large impact but it is well contained under the MSI
support. Initial tests were done on the P8, P9 and P10 PowerNV and
pSeries platforms, under the KVM and PowerVM hypervisor. PCI passthrough
was tested on P8/KVM, P9/KVM and P9/pVM with both interrupt modes.
P8 passthrough has some optimization to EOI MSIs when under real mode :
e3c13e56a471 ("KVM: PPC: Book3S HV: Handle passthrough interrupts in guest")
5d375199ea96 ("KVM: PPC: Book3S HV: Set server for passed-through interrupts")
They give us a ~10% bandwidth improvement on some 100G adapters
(Thanks Alexey), so it's good to keep but they require access to the
low level IRQ domain of the machine. It should be possible to rework
the code and use the MSI IRQ domains instead but for now, it's simpler
to keep the bypass. That can come later.
The P8/CAPI driver is also impacted. Tests were done on a Firestone
system with a memory AFU.
Thanks,
C.
Changes since v2 :
- Included some CONFIG_IRQ_DOMAIN_HIERARCHY ifdefs
- Microwatt fixes for ICS native
- Removed irqd_is_started() check when setting the affinity
Cédric Le Goater (32):
powerpc/pseries/pci: Introduce __find_pe_total_msi()
powerpc/pseries/pci: Introduce rtas_prepare_msi_irqs()
powerpc/xive: Add support for IRQ domain hierarchy
powerpc/xive: Ease debugging of xive_irq_set_affinity()
powerpc/pseries/pci: Add MSI domains
powerpc/xive: Drop unmask of MSIs at startup
powerpc/xive: Remove irqd_is_started() check when setting the affinity
powerpc/pseries/pci: Add a domain_free_irqs() handler
powerpc/pseries/pci: Add a msi_free() handler to clear XIVE data
powerpc/pseries/pci: Add support of MSI domains to PHB hotplug
powerpc/powernv/pci: Introduce __pnv_pci_ioda_msi_setup()
powerpc/powernv/pci: Add MSI domains
KVM: PPC: Book3S HV: Use the new IRQ chip to detect passthrough
interrupts
KVM: PPC: Book3S HV: XIVE: Change interface of passthrough interrupt
routines
KVM: PPC: Book3S HV: XIVE: Fix mapping of passthrough interrupts
powerpc/xics: Remove ICS list
powerpc/xics: Rename the map handler in a check handler
powerpc/xics: Give a name to the default XICS IRQ domain
powerpc/xics: Add debug logging to the set_irq_affinity handlers
powerpc/xics: Add support for IRQ domain hierarchy
powerpc/powernv/pci: Customize the MSI EOI handler to support PHB3
powerpc/pci: Drop XIVE restriction on MSI domains
powerpc/xics: Drop unmask of MSIs at startup
powerpc/pseries/pci: Drop unused MSI code
powerpc/powernv/pci: Drop unused MSI code
powerpc/powernv/pci: Adapt is_pnv_opal_msi() to detect passthrough
interrupt
powerpc/xics: Fix IRQ migration
powerpc/powernv/pci: Set the IRQ chip data for P8/CXL devices
powerpc/powernv/pci: Rework pnv_opal_pci_msi_eoi()
KVM: PPC: Book3S HV: XICS: Fix mapping of passthrough interrupts
powerpc/xive: Use XIVE domain under xmon and debugfs
genirq: Improve "hwirq" output in /proc and /sys/
arch/powerpc/include/asm/kvm_ppc.h | 4 +-
arch/powerpc/include/asm/pci-bridge.h | 5 +
arch/powerpc/include/asm/pnv-pci.h | 2 +-
arch/powerpc/include/asm/xics.h | 3 +-
arch/powerpc/include/asm/xive.h | 1 +
arch/powerpc/platforms/powernv/pci.h | 6 -
arch/powerpc/platforms/pseries/pseries.h | 2 +
arch/powerpc/kernel/pci-common.c | 6 +
arch/powerpc/kvm/book3s_hv.c | 18 +-
arch/powerpc/kvm/book3s_hv_rm_xics.c | 8 +-
arch/powerpc/kvm/book3s_xive.c | 18 +-
arch/powerpc/platforms/powernv/pci-ioda.c | 256 ++++++++++++++++--
arch/powerpc/platforms/powernv/pci.c | 67 -----
arch/powerpc/platforms/pseries/msi.c | 296 ++++++++++++++++-----
arch/powerpc/platforms/pseries/pci_dlpar.c | 4 +
arch/powerpc/platforms/pseries/setup.c | 2 +
arch/powerpc/sysdev/xics/ics-native.c | 13 +-
arch/powerpc/sysdev/xics/ics-opal.c | 40 +--
arch/powerpc/sysdev/xics/ics-rtas.c | 40 +--
arch/powerpc/sysdev/xics/xics-common.c | 129 ++++++---
arch/powerpc/sysdev/xive/common.c | 98 +++++--
kernel/irq/irqdesc.c | 2 +-
kernel/irq/irqdomain.c | 1 +
kernel/irq/proc.c | 2 +-
24 files changed, 710 insertions(+), 313 deletions(-)
--
2.31.1
