On Thu, 14 Sep 2017 19:39:40 +0000 (GMT) Allen Dial <ad...@mac.com> wrote:
> Hi VFIO-Users, > > I have 2 of 3 2 port NICs that I would like to passthrough to guest vms but > am not sure how best to go about it as 1, they get divided up (not the > biggest issue other than I don't know how I will tell which one is which > physically out the back of the machine) but more importantly they each get > grouped with pci bridges in their iommu groups. > > Is it possible to pass 4 of the interfaces to vms and keep 2 of them for the > host? > > Below is some hopefully helpful information from the terminal: > > group 39 > /sys/kernel/iommu_groups/39/devices/0000:42:02.0 > 42:02.0 PCI bridge: Integrated Device Technology, Inc. [IDT] PES24T3G2 PCI > Express Gen2 Switch (rev 02) > /sys/kernel/iommu_groups/39/devices/0000:43:00.0 > 43:00.0 Ethernet controller: MYRICOM Inc. Myri-10G Dual-Protocol NIC (rev 01) > > group 40 > /sys/kernel/iommu_groups/40/devices/0000:42:04.0 > 42:04.0 PCI bridge: Integrated Device Technology, Inc. [IDT] PES24T3G2 PCI > Express Gen2 Switch (rev 02) > /sys/kernel/iommu_groups/40/devices/0000:44:00.0 > 44:00.0 Ethernet controller: MYRICOM Inc. Myri-10G Dual-Protocol NIC (rev 01) Looks like the card likely includes a PCIe switch where the downstream ports do not support ACS, so they get grouped with the endpoint below it. The real question is probably whether the downstream ports 42:02.0 and 42:04.0 should have been grouped together, putting both downstream ports and both endpoints in the same group. That probably didn't happen because the downstream ports have different "slot" addresses. > group 42 > /sys/kernel/iommu_groups/42/devices/0000:47:02.0 > 47:02.0 PCI bridge: Integrated Device Technology, Inc. [IDT] PES24T3G2 PCI > Express Gen2 Switch (rev 02) > /sys/kernel/iommu_groups/42/devices/0000:48:00.0 > 48:00.0 Ethernet controller: MYRICOM Inc. Myri-10G Dual-Protocol NIC (rev 01) > > group 43 > /sys/kernel/iommu_groups/43/devices/0000:47:04.0 > 47:04.0 PCI bridge: Integrated Device Technology, Inc. [IDT] PES24T3G2 PCI > Express Gen2 Switch (rev 02) > /sys/kernel/iommu_groups/43/devices/0000:49:00.0 > 49:00.0 Ethernet controller: MYRICOM Inc. Myri-10G Dual-Protocol NIC (rev 01) Set #2 > group 45 > /sys/kernel/iommu_groups/45/devices/0000:4b:02.0 > 4b:02.0 PCI bridge: Integrated Device Technology, Inc. [IDT] PES24T3G2 PCI > Express Gen2 Switch (rev 02) > /sys/kernel/iommu_groups/45/devices/0000:4c:00.0 > 4c:00.0 Ethernet controller: MYRICOM Inc. Myri-10G Dual-Protocol NIC (rev 01) > > group 46 > /sys/kernel/iommu_groups/46/devices/0000:4b:04.0 > 4b:04.0 PCI bridge: Integrated Device Technology, Inc. [IDT] PES24T3G2 PCI > Express Gen2 Switch (rev 02) > /sys/kernel/iommu_groups/46/devices/0000:4d:00.0 > 4d:00.0 Ethernet controller: MYRICOM Inc. Myri-10G Dual-Protocol NIC (rev 01) Set #3 As it is, there's no problem with assigning each NIC to a separate guest or using them in the host. It is arguable whether there should really only be 3 groups here as peer-to-peer DMA between downstream switch ports seems possible in this configuration. If we fix that then you'd need to assign NICs in sets of two. As it is, each NIC is assignable, for better or worse. Thanks, Alex _______________________________________________ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users