Hi, This is v7 of the server side changes to enable vfio-user in QEMU.
Thank you very much for reviewing the last revision of this series! In this revision, we've dropped the patches concerning migration due to a major changes in the VFIO PCI protocol. We are changing the client, library and server to conform with the updated spec, and will send the migration patches for review separately. The review of client side changes are happening parallelly [1]. The avocado test in Patch 17 of this series depends on the client, as such it has to wait until client changes are available. However, we run the avocado test, among others, to confirm that the server is working as expected - we do it before submitting patches for review each time. The following repo applies this series on the latest client, in case anyone would like to run vfio-user. repo: https://github.com/oracle/qemu branch: vfio-user-client-v7server launcher: scripts/vfiouser-launcher.py [1]: https://lore.kernel.org/all/6d8ae21cbade8f4bb7eaca4da29e57f0cb1a03f3.1641584317.git.john.g.john...@oracle.com/T/ We've made the following changes in this revision: [PATCH v7 02/17] qdev: unplug blocker for devices - corrects comments to prevent creation of new section for unplug blocker - adds an assert to device_finalize() to confirm that device does not have unplug blockers - moves the unplug blocker functions to hw/core/qdev.c - moves test for unplug blocker to qdev_unplug() from qmp_device_del() [PATCH v7 05/17] configure: require cmake 3.19 or newer - new in this series [PATCH v7 06/17] vfio-user: build library - configure script sets cmake_required flag for vfio user [PATCH v7 07/17] vfio-user: define vfio-user-server object - adds auto-shutdown sub-option to the remote machine - adds boolean auto_shutdown to TYPE_REMOTE_MACHINE's class - adds vfu_object_auto_shutdown() helper function to vfio-user-obj.c to query the auto-shutdown property from the machine - reworks VFU_OBJECT_ERROR() & vfu_object_finalize() to use the helper function above. - updates QEMU version to 7.1 in commentary for VfioUserServerProperties [PATCH v7 08/17] vfio-user: instantiate vfio-user context - moves phase_check() after the check for machine type in vfu_object_init() - sets o->vfu_ctx to NULL in vfu_object_finalize() [PATCH v7 09/17] vfio-user: find and init PCI device - holds a reference to attached device in the server, and unrefs it during cleanup [PATCH v7 10/17] vfio-user: run vfio-user context - updates QEMU version to 7.1 in commentary for VFU_CLIENT_HANGUP [PATCH v7 11/17] vfio-user: handle PCI config space accesses - adds a comment explaining that writes to BAR register in config space doesn't create conflicting memory regions [PATCH v7 12/17] vfio-user: IOMMU support for remote device - passes IOMMU table as an opaque data to pci_setup_iommu() - adds locking to access the table to enable concurrent access - retains the global data structure as remote_iommu_del_device() needs it because it doesn't have access to the opaque data - removes redundant check to confirm if hash table is present in remote_iommu_find_add_as() - adds remote_iommu_del_elem() to free IOMMU entry when it is removed from the table [PATCH v7 14/17] vfio-user: handle PCI BAR accesses - adjusts the 'offset' in vfu_object_bar_rw() - MemoryRegion returned by memory_region_find() could be a subregion of the root memory region referenced by pci_dev->io_regions[pci_bar].memory. 'offset' input argument is relative to the root region, whereas it must be relative to the subregion before access. - adds warning for out-of-range access [PATCH v7 15/17] vfio-user: handle device interrupts - moves msi_nonbroken initialization to hw/remote/machine.c - adds irq_opaque to PCIDevice which the interrupt notification could use; drops global hash table which map device to vfio-user context - removes NULL function pointer test in msi_notify and msix_notify - vfu_object_msi_notify() asserts that IRQ vector is allocated [PATCH v7 16/17] vfio-user: handle reset of remote device - adds comment to explain lost connection handling [PATCH v7 17/17] vfio-user: avocado tests for vfio-user - drops the migration test Dropped the following patches: configure, meson: override C compiler for cmake softmmu/vl: defer backend init vfio-user: register handlers to facilitate migration We are looking forward to your comments. Thank you very much! Jagannathan Raman (17): tests/avocado: Specify target VM argument to helper routines qdev: unplug blocker for devices remote/machine: add HotplugHandler for remote machine remote/machine: add vfio-user property configure: require cmake 3.19 or newer vfio-user: build library vfio-user: define vfio-user-server object vfio-user: instantiate vfio-user context vfio-user: find and init PCI device vfio-user: run vfio-user context vfio-user: handle PCI config space accesses vfio-user: IOMMU support for remote device vfio-user: handle DMA mappings vfio-user: handle PCI BAR accesses vfio-user: handle device interrupts vfio-user: handle reset of remote device vfio-user: avocado tests for vfio-user configure | 36 +- meson.build | 44 +- qapi/misc.json | 23 + qapi/qom.json | 20 +- include/exec/memory.h | 3 + include/hw/pci/pci.h | 10 + include/hw/qdev-core.h | 29 + include/hw/remote/iommu.h | 18 + include/hw/remote/machine.h | 10 +- include/hw/remote/vfio-user-obj.h | 6 + hw/core/qdev.c | 24 + hw/pci/msi.c | 11 +- hw/pci/msix.c | 10 +- hw/remote/iommu.c | 95 +++ hw/remote/machine.c | 73 +- hw/remote/vfio-user-obj.c | 847 +++++++++++++++++++++ softmmu/physmem.c | 4 +- softmmu/qdev-monitor.c | 4 + stubs/vfio-user-obj.c | 6 + tests/qtest/fuzz/generic_fuzz.c | 9 +- .gitlab-ci.d/buildtest.yml | 2 + .gitmodules | 3 + Kconfig.host | 4 + MAINTAINERS | 6 + hw/remote/Kconfig | 4 + hw/remote/meson.build | 4 + hw/remote/trace-events | 11 + meson_options.txt | 3 + stubs/meson.build | 1 + subprojects/libvfio-user | 1 + tests/avocado/avocado_qemu/__init__.py | 14 +- tests/avocado/vfio-user.py | 164 ++++ tests/docker/dockerfiles/centos8.docker | 2 + tests/docker/dockerfiles/ubuntu2004.docker | 2 + 34 files changed, 1483 insertions(+), 20 deletions(-) create mode 100644 include/hw/remote/iommu.h create mode 100644 include/hw/remote/vfio-user-obj.h create mode 100644 hw/remote/iommu.c create mode 100644 hw/remote/vfio-user-obj.c create mode 100644 stubs/vfio-user-obj.c create mode 160000 subprojects/libvfio-user create mode 100644 tests/avocado/vfio-user.py -- 2.20.1
