On 12/08/2014 06:04 PM, Alex Williamson wrote: > On Sun, 2014-11-30 at 18:35 +0000, Eric Auger wrote: >> This RFC series aims at enabling KVM platform device passthrough. >> It implements a VFIO platform device, derived from VFIO PCI device. >> >> The VFIO platform device uses the host VFIO platform driver which must >> be bound to the assigned device prior to the QEMU system start. >> >> - the guest can directly access the device register space >> - assigned device IRQs are transparently routed to the guest by >> QEMU/KVM (3 methods currently are supported: user-level eventfd >> handling, irqfd, forwarded IRQs) >> - iommu is transparently programmed to prevent the device from >> accessing physical pages outside of the guest address space >> >> This patch series is made of the following patch file groups: >> >> 1-11) PCI modifications to prepare for platform device introduction > > Hi Eric, > > Sorry for the delay, I've only been able to make a serious review of > 1-11 and it looks good. I think we should start trying to pull these in > after QEMU2.2.
That's a great news! Looking beyond patch 11/19, I'm surprised that the very > first next patch isn't a linux header sync so that the platform > populate_device routine can test the device info to make sure it's > operating on a vfio platform device. Yes you're fully right. Currently I do not test it and it's a mistake. I will add this linux header sync. Thanks Eric Thanks, > > Alex > >> 12-15) VFIO & calxeda midway platform device without irqfd support >> 16) VFIO platform device with irqfd support >> 17-19) VFIO platform device with IRQ forwarding support >> >> Each group is independent and should be separately upstreamable. >> >> Dependency List: >> >> QEMU dependencies: >> [1] [PATCH v5] machvirt dynamic sysbus device instantiation >> Eric Auger >> [2] [PATCH v3 0/2] actual checks of KVM_CAP_IRQFD and KVM_CAP_IRQFD_RESAMPLE, >> Eric Auger >> http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00589.html >> [3] [PATCH v2] vfio: migration to trace points >> Eric Auger >> https://patchwork.ozlabs.org/patch/394785/ >> >> Kernel Dependencies: >> [1] [PATCH v10 00/20] VFIO support for platform and AMBA devices on ARM >> Antonios Motakis >> http://comments.gmane.org/gmane.linux.kernel.iommu/7096 >> [2] [PATCH v3 0/6] vfio: type1: support for ARM SMMUS with VFIO_IOMMU_TYPE1 >> Antonios Motakis >> http://www.spinics.net/lists/kvm-arm/msg11738.html >> [3] [PATCH v4] ARM: KVM: add irqfd support >> Eric Auger >> https://lkml.org/lkml/2014/9/1/141 >> [4] [RFC PATCH 0/9] ARM: Forwarding physical interrupts to a guest VM, >> Marc Zyngier >> http://lwn.net/Articles/603514/ >> [5] [PATCH v3 0/9] KVM-VFIO IRQ forward control >> Eric Auger >> https://lkml.org/lkml/2014/9/1/344 >> >> - kernel pieces can be found at: >> http://git.linaro.org/people/eric.auger/linux.git (branch 3.18-rc6-v10) >> - QEMU pieces can be found at: >> http://git.linaro.org/people/eric.auger/qemu.git (branch vfio_integ_v8) >> >> The patch series was tested on Calxeda Midway (ARMv7) where one xgmac >> is assigned to KVM host while the second one is assigned to the guest. >> Reworked PCI device is not tested. >> >> Wiki for Calxeda Midway setup: >> https://wiki.linaro.org/LEG/Engineering/Virtualization/Platform_Device_Passthrough_on_Midway >> >> History: >> v7->v8: >> - rebase on v2.2.0-rc3 and integrate >> "Add skip_dump flag to ignore memory region during dump" >> - KVM header evolution with subindex addition in kvm_arch_forwarded_irq >> - split [PATCH v7 03/16] hw/vfio/pci: introduce VFIODevice into 4 patches >> - vfio_compute_needs_reset does not return bool anymore >> - add some comments about exposed MMIO region and IRQ in calxeda xgmac >> device >> - vfio_[un]mask_irqindex renamed into vfio_[un]mask_single_irqindex >> - rework IRQ startup: former machine init done notifier is replaced by a >> reset notifier. machine file passes the interrupt controller >> DeviceState handle (not the platform bus first irq parameter). >> - sysbus-fdt: >> - move the add_fdt_node_functions array declaration between the device >> specific code and the generic code to avoid forward declarations of >> decice specific functions >> - rename add_basic_vfio_fdt_node into add_calxeda_midway_xgmac_fdt_node >> emphasizing the fact it is xgmac specific >> >> v6->v7: >> - fake injection test modality removed >> - VFIO_DEVICE_TYPE_PLATFORM only introduced with VFIO platform >> - new helper functions to start VFIO IRQ on machine init done notifier >> (introduced in hw/vfio/platform: add vfio-platform support and notifier >> registration invoked in hw/arm/virt: add support for VFIO devices). >> vfio_start_irq_injection is replaced by vfio_register_irq_starter. >> >> v5->v6: >> - rebase on 2.1rc5 PCI code >> - forwarded IRQ first integraton >> - vfio_device property renamed into host property >> - split IRQ setup in different functions that match the 3 supported >> injection techniques (user handled eventfd, irqfd, forwarded IRQ): >> removes dynamic switch between injection methods >> - introduce fake interrupts as a test modality: >> x makes possible to test multiple IRQ user-side handling. >> x this is a test feature only: enable to trigger a fd as if the >> real physical IRQ hit. No virtual IRQ is injected into the guest >> but handling is simulated so that the state machine can be tested >> - user handled eventfd: >> x add mutex to protect IRQ state & list manipulation, >> x correct misleading comment in vfio_intp_interrupt. >> x Fix bugs using fake interrupt modality >> - irqfd no more advertised in this patchset (handled in [3]) >> - VFIOPlatformDeviceClass becomes abstract and Calxeda xgmac device >> and class is re-introduced (as per v4) >> - all DPRINTF removed in platform and replaced by trace-points >> - corrects compilation with configure --disable-kvm >> - simplifies the split for vfio_get_device and introduce a unique >> specialized function named vfio_populate_device >> - group_list renamed into vfio_group_list >> - hw/arm/dyn_sysbus_devtree.c currently only support vfio-calxeda-xgmac >> instantiation. Needs to be specialized for other VFIO devices >> - fix 2 bugs in dyn_sysbus_devtree(reg_attr index and compat) >> >> v4->v5: >> - rebase on v2.1.0 PCI code >> - take into account Alex Williamson comments on PCI code rework >> - trace updates in vfio_region_write/read >> - remove fd from VFIORegion >> - get/put ckeanup >> - bug fix: bar region's vbasedev field duly initialization >> - misc cleanups in platform device >> - device tree node generation removed from device and handled in >> hw/arm/dyn_sysbus_devtree.c >> - remove "hw/vfio: add an example calxeda_xgmac": with removal of >> device tree node generation we do not have so many things to >> implement in that derived device yet. May be re-introduced later >> on if needed typically for reset/migration. >> - no GSI routing table anymore >> >> v3->v4 changes (Eric Auger, Alvise Rigo) >> - rebase on last VFIO PCI code (v2.1.0-rc0) >> - full git history rework to ease PCI code change review >> - mv include files in hw/vfio >> - DPRINTF reformatting temporarily moved out >> - support of VFIO virq (removal of resamplefd handler on user-side) >> - integration with sysbus dynamic instantiation framwork >> - removal of unrealize and cleanup routines until it is better >> understood what is really needed >> - Support of VFIO for Amba devices should be handled in an inherited >> device to specialize the device tree generation (clock handle currently >> missing in framework however) >> - "Always use eventfd as notifying mechanism" temporarily moved out >> - static instantiation is not mainstream (although it remains possible) >> note if static instantiation is used, irqfd must be setup in machine file >> when virtual IRQ is known >> - create the GSI routing table on qemu side >> >> v2->v3 changes (Alvise Rigo, Eric Auger): >> - Following Alex W recommandations, further efforts to factorize the >> code between PCI:introduction of VFIODevice and VFIORegion >> as base classes >> - unique reset handler for platform and PCI >> - cleanup following Kim's comments >> - multiple IRQ support mechanics should be in place although not >> tested >> - Better handling of MMIO multiple regions >> - New features and fixes by Alvise (multiple compat string, exec >> flag, force eventfd usage, amba device tree support) >> - irqfd support >> >> v1->v2 changes (Kim Phillips, Eric Auger): >> - IRQ initial support (legacy mode where eventfds are handled on >> user side) >> - hacked dynamic instantiation >> >> v1 (Kim Phillips): >> - initial split between PCI and platform >> - MMIO support only >> - static instantiation >> >> Best Regards >> >> Eric >> >> >> >> >> Eric Auger (18): >> hw/vfio/pci: Rename VFIODevice into VFIOPCIDevice >> hw/vfio/pci: generalize mask/unmask to any IRQ index >> hw/vfio/pci: introduce minimalist VFIODevice with fd >> hw/vfio/pci: add type, name and group fields in VFIODevice >> hw/vfio/pci: handle reset at VFIODevice >> hw/vfio/pci: Introduce VFIORegion >> hw/vfio/pci: split vfio_get_device >> hw/vfio/pci: rename group_list into vfio_group_list >> hw/vfio/pci: use name field in format strings >> hw/vfio: create common module >> hw/vfio/platform: add vfio-platform support >> hw/vfio: calxeda xgmac device >> hw/arm/virt: add support for VFIO devices >> hw/arm/sysbus-fdt: enable vfio-calxeda-xgmac dynamic instantiation >> hw/vfio/platform: Add irqfd support >> linux-headers: Update KVM headers from linux-next tag ToBeFilled >> hw/vfio/common: vfio_kvm_device_fd moved in the common header >> hw/vfio/platform: add forwarded irq support >> >> Kim Phillips (1): >> vfio: move hw/misc/vfio.c to hw/vfio/pci.c Move vfio.h into >> include/hw/vfio >> >> LICENSE | 2 +- >> MAINTAINERS | 2 +- >> hw/Makefile.objs | 1 + >> hw/arm/sysbus-fdt.c | 88 ++ >> hw/arm/virt.c | 15 +- >> hw/misc/Makefile.objs | 1 - >> hw/ppc/spapr_pci_vfio.c | 2 +- >> hw/vfio/Makefile.objs | 6 + >> hw/vfio/calxeda_xgmac.c | 54 ++ >> hw/vfio/common.c | 960 +++++++++++++++++++ >> hw/{misc/vfio.c => vfio/pci.c} | 1671 >> +++++++--------------------------- >> hw/vfio/platform.c | 777 ++++++++++++++++ >> include/hw/vfio/vfio-calxeda-xgmac.h | 46 + >> include/hw/vfio/vfio-common.h | 157 ++++ >> include/hw/vfio/vfio-platform.h | 88 ++ >> include/hw/{misc => vfio}/vfio.h | 0 >> linux-headers/linux/kvm.h | 10 + >> trace-events | 137 +-- >> 18 files changed, 2599 insertions(+), 1418 deletions(-) >> create mode 100644 hw/vfio/Makefile.objs >> create mode 100644 hw/vfio/calxeda_xgmac.c >> create mode 100644 hw/vfio/common.c >> rename hw/{misc/vfio.c => vfio/pci.c} (65%) >> create mode 100644 hw/vfio/platform.c >> create mode 100644 include/hw/vfio/vfio-calxeda-xgmac.h >> create mode 100644 include/hw/vfio/vfio-common.h >> create mode 100644 include/hw/vfio/vfio-platform.h >> rename include/hw/{misc => vfio}/vfio.h (100%) >> > > >
