This patch set consists of two parts:
- The two first patches implements SR/IOV building blocks in pcie/pci.
I have held this patch back for a while because I haven't had a good
example test case to accompany it, but in the light of the latest
developments such as the discussion we have had around ARI and downstream
switches and root ports, I think it would be valuable to get this in,
to make it easier for people to experiment with creating devices with
many functions. Hopefully, I can also get some help to fix the hotplug
issues described below.
- The two last patches contains an example to illustrate the usage
of the new SR/IOV support. The example leverages the e1000
code and the fact that registers between E1000 and the PCIe based
Intel 82576 Gigabit Ethernet Adapter (which supports SR/IOV) are quite
similar,
but so far without considering much of the differences beyond the
bare minimum needed to trick the igb driver into loading to the point
where VFs can be enabled.
So you cannot yet use these PF or VF Ethernet devices to send ethernet
packets,
and the implementation do not in any way attempt to model the internals
of igb such as the multiple queues or any multiplexing onto the same device,
it only instantiates the VFs as well as the PFs mostly directly by
inheritance
(and as separate devices) from E1000, but it should hopefully be relatively
easy to understand how to proceed to make "true" VFs.
It was also a nice exercise in using QOM.
The changes to E1000 to accommodate igb (patch 3) should be fairly
non-intrusive, nevertheless I suppose it should not be applied
unless it will eventually lead to a new derived device which is enough
different from E1000 to qualify for a separate set of source files.
So if someone with more detailed knowledge of the internal differences
between igb and e1000 on the functional level might have input, that would be
great, an alternative of course be to only apply the two first
patches, and leave any usage examples for the future.
To test and see how it plays out, you can add something like this
to the command line:
-device ioh3420,slot=0,id=pcie_port.0
-device igb,mac=DE:AD:BE:EE:04:18,vlan=1,bus=pcie_port.0
-net tap,vlan=1
The Linux igb driver does not yet support changing the VF setup via sysfs
(the preferred way since kernel v.3.8) so to see some VFs on the guest,
you need to set up a modprobe file with something like this:
options igb max_vfs=4
For some reason I dont yet understand, removal of the igb driver
does not cause a 0 write to sr/iov vf_enable to disable the VFs again, eg.
# lspci | grep '^01:00'
01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82576 Virtual Function
(rev 01)
01:00.2 Ethernet controller: Intel Corporation 82576 Virtual Function
(rev 01)
01:00.3 Ethernet controller: Intel Corporation 82576 Virtual Function
(rev 01)
01:00.4 Ethernet controller: Intel Corporation 82576 Virtual Function
(rev 01)
Writing directly to vf_enable using setpci works though, but after the later
hotplug
changes, a little too well, as after the VF removals have succeded,
some way a slot power down gets triggered, also removing the PF:
# setpci -s 01:00.0 168.b
0x19
# setpci -s 01:00.0 168.b=18
# lspci | grep '^01:00'
<nothing>
For similar reasons attaching the igb device directly on the root complex
(just remove
bus parameter above) attempts to enable VFs will fail with
qdev.c:89: bus_add_child: Assertion `bus->allow_hotplug' failed
I imagine this could for instance be a matter of defining a property
"subfunction"
or similar that allows qdev (QOM?) to discriminate between main devices and
devices
representing subfunctions of another main device. Suggestions on how to
proceed on this
welcome.
This patch depends (by diff context only) on my patch:
[PATCH v2 1/4] pcie: Fix incorrect write to the ari capability next function
field
and for stability on Paolo's
[PATCH] pci_bridge: manually destroy memory regions within PCIBridgeWindows
both of which Michael has pulled, but which are not in master yet.
Thanks,
Knut Omang (4):
pci: Update pci_regs header
pcie: Add support for Single Root I/O Virtualization (SR/IOV)
e1000: Refactor to allow subclassing from other source file
igb: Example code to illustrate the SR/IOV support.
hw/i386/kvm/pci-assign.c | 4 +-
hw/misc/vfio.c | 8 +-
hw/net/Makefile.objs | 2 +-
hw/net/e1000.c | 126 ++--------------
hw/net/e1000.h | 139 +++++++++++++++++
hw/net/igb.c | 293 ++++++++++++++++++++++++++++++++++++
hw/net/igb_regs.h | 27 ++++
hw/pci/msi.c | 4 -
hw/pci/msix.c | 2 +-
hw/pci/pci.c | 107 +++++++++----
hw/pci/pcie.c | 205 ++++++++++++++++++++++++-
include/hw/pci/pci.h | 6 +-
include/hw/pci/pci_regs.h | 371 ++++++++++++++++++++++++++++++++++------------
include/hw/pci/pcie.h | 26 ++++
14 files changed, 1072 insertions(+), 248 deletions(-)
create mode 100644 hw/net/e1000.h
create mode 100644 hw/net/igb.c
create mode 100644 hw/net/igb_regs.h
--
1.9.0
