Even while I use SR-IOV interfaces passed through, I still find
openvswitch very useful.
In your scenario I might just use openvswitch and bind all the NICs.
Perhaps use DPDK if you're primarily doing networking with the device.
Also CoreOS might be nice if again you're building a dedicated
router/firewall.
On 26 Mar 2016, at 4:28, YAEGASHI Takeshi wrote:
Hello,
I have a fitlet http://www.fit-pc.com/web/products/fitlet/ with 4 GbE
(I211) ports and wanted to passthrough some of them to KVM guests, ie
assign one I211 to each guests. I've tried various configurations
with libvirt/kvm but no luck so far.
After reading
http://vfio.blogspot.jp/2014/08/iommu-groups-inside-and-out.html
I've got sure that there's a problem in IOMMU groups. Actually all of
I211 and AMD's PCI bridge [1022:156b] share the same iommu_group 2.
$ lspci -tv
-[0000:00]-+-00.0 Advanced Micro Devices, Inc. [AMD] Device 1566
+-00.2 Advanced Micro Devices, Inc. [AMD] Device 1567
+-01.0 Advanced Micro Devices, Inc. [AMD/ATI] Mullins
[Radeon R6 Graphics]
+-01.1 Advanced Micro Devices, Inc. [AMD/ATI] Kabini
HDMI/DP Audio
+-02.0 Advanced Micro Devices, Inc. [AMD] Device 156b
+-02.2-[01]----00.0 Intel Corporation I211 Gigabit Network
Connection
+-02.3-[02]----00.0 Intel Corporation I211 Gigabit Network
Connection
+-02.4-[03]----00.0 Intel Corporation I211 Gigabit Network
Connection
+-02.5-[04]----00.0 Intel Corporation I211 Gigabit Network
Connection
+-08.0 Advanced Micro Devices, Inc. [AMD] Device 1537
+-10.0 Advanced Micro Devices, Inc. [AMD] FCH USB XHCI
Controller
+-11.0 Advanced Micro Devices, Inc. [AMD] FCH SATA
Controller [AHCI mode]
+-12.0 Advanced Micro Devices, Inc. [AMD] FCH USB EHCI
Controller
+-13.0 Advanced Micro Devices, Inc. [AMD] FCH USB EHCI
Controller
+-14.0 Advanced Micro Devices, Inc. [AMD] FCH SMBus
Controller
+-14.2 Advanced Micro Devices, Inc. [AMD] FCH Azalia
Controller
+-14.3 Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge
+-14.7 Advanced Micro Devices, Inc. [AMD] FCH SD Flash
Controller
+-18.0 Advanced Micro Devices, Inc. [AMD] Device 1580
+-18.1 Advanced Micro Devices, Inc. [AMD] Device 1581
+-18.2 Advanced Micro Devices, Inc. [AMD] Device 1582
+-18.3 Advanced Micro Devices, Inc. [AMD] Device 1583
+-18.4 Advanced Micro Devices, Inc. [AMD] Device 1584
\-18.5 Advanced Micro Devices, Inc. [AMD] Device 1585
$ find /sys/kernel/iommu_groups -type l
/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/1/devices/0000:00:01.1
/sys/kernel/iommu_groups/2/devices/0000:00:02.0
/sys/kernel/iommu_groups/2/devices/0000:00:02.2
/sys/kernel/iommu_groups/2/devices/0000:00:02.3
/sys/kernel/iommu_groups/2/devices/0000:00:02.4
/sys/kernel/iommu_groups/2/devices/0000:00:02.5
/sys/kernel/iommu_groups/2/devices/0000:01:00.0
/sys/kernel/iommu_groups/2/devices/0000:02:00.0
/sys/kernel/iommu_groups/2/devices/0000:03:00.0
/sys/kernel/iommu_groups/2/devices/0000:04:00.0
/sys/kernel/iommu_groups/3/devices/0000:00:08.0
/sys/kernel/iommu_groups/4/devices/0000:00:10.0
/sys/kernel/iommu_groups/5/devices/0000:00:11.0
/sys/kernel/iommu_groups/6/devices/0000:00:12.0
/sys/kernel/iommu_groups/7/devices/0000:00:13.0
/sys/kernel/iommu_groups/8/devices/0000:00:14.0
/sys/kernel/iommu_groups/8/devices/0000:00:14.2
/sys/kernel/iommu_groups/8/devices/0000:00:14.3
/sys/kernel/iommu_groups/8/devices/0000:00:14.7
/sys/kernel/iommu_groups/9/devices/0000:00:18.0
/sys/kernel/iommu_groups/9/devices/0000:00:18.1
/sys/kernel/iommu_groups/9/devices/0000:00:18.2
/sys/kernel/iommu_groups/9/devices/0000:00:18.3
/sys/kernel/iommu_groups/9/devices/0000:00:18.4
/sys/kernel/iommu_groups/9/devices/0000:00:18.5
I'm running Ubuntu 14.04 with xenial kernel 4.4.0-13, libvirt 1.2.2,
qemu 2.0.0. Using vfio-pci simply failed:
qemu-system-x86_64: -device
vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x3: vfio: error,
group 2 is not viable, please ensure all devices within the
iommu_group are bound to their vfio bus driver.
qemu-system-x86_64: -device
vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x3: vfio: failed to
get group 2
qemu-system-x86_64: -device
vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x3: Device
initialization failed.
qemu-system-x86_64: -device
vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x3: Device
'vfio-pci' could not be initialized
Using legacy KVM device assignment with <driver name='kvm'/> also
failed with unclear reason "Invalid argument":
qemu-system-x86_64: -device
pci-assign,configfd=24,host=01:00.0,id=hostdev0,bus=pci.2,addr=0x2:
Failed to assign device "hostdev0" : Invalid argument
qemu-system-x86_64: -device
pci-assign,configfd=24,host=01:00.0,id=hostdev0,bus=pci.2,addr=0x2:
Device initialization failed.
qemu-system-x86_64: -device
pci-assign,configfd=24,host=01:00.0,id=hostdev0,bus=pci.2,addr=0x2:
Device 'kvm-pci-assign' could not be initialized
After some googling I've learned that legacy KVM assignment also
refuses to work for this configuration since kernel 4.2
https://lkml.org/lkml/2015/11/12/661 . With older kernel 3.19 this
configuration worked as expected.
Is there any chance for me to achieve single I211 passthrough with
recent kernels? Because the hardware has no flexibility on the bus
topologies, the only possible way would be patching the kernel (ACS
override or PCI quirks). Is it safe? Does anyone have any info on
the IOMMU capability of device/bridge/chipset in question?
Regards,
--
YAEGASHI Takeshi <yaega...@debian.org>
_______________________________________________
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users
_______________________________________________
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users
