On Wed, 5 Jul 2017 02:33:52 -0400 "taii...@gmx.com" <taii...@gmx.com> wrote:
> error: internal error: Process exited prior to exec: libvirt: error : > internal error: Invalid device 0000:09:00.1 iommu_group file > /sys/bus/pci/devices/0000:09:00.1/iommu_group is not a symlink > > (intel 82576 quad port nic, assigning itself works fine) > > I am trying to get sr-iov working on a platform with 32bit MMIO space > (is that even possible) > > So I have added pci-realloc and pci-assign-buses to the kernel command > line, without that I get a "Not enough MMIO resources for SR-IOV" error > even when adding just one VF via sysfs. This is not really a 32-bit vs 64-bit issue, it just means your BIOS didn't allocate enough resources on the bus to enable SR-IOV. The 82576 dual-port cards typically get away with working on non-SR-IOV aware systems because the additional resources they need for SR-IOV fits within the minimum bridge apertures anyway. This is not true for the quad-port card or various other SR-IOV devices. pci-realloc should help with this, whether pci-assign-buses is necessary would depend on the SR-IOV config of the device (82576 typically doesn't need an additional bus number). > I also get an error about not enough space for BAR's in dmesg with or > without the cmdline additions but with the cmdline parms the vf bars > seem to allocate fine. > > [ 2.705747] pci 0000:07:00.0: VF(n) BAR0 space: [mem > 0x00000000-0x0001ffff 64bit] (contains BAR0 for 8 VFs) > [ 2.715887] pci 0000:07:00.0: VF(n) BAR3 space: [mem > 0x00000000-0x0001ffff 64bit] (contains BAR3 for 8 VFs) > [ 2.726342] pci 0000:07:00.1: VF(n) BAR0 space: [mem > 0x00000000-0x0001ffff 64bit] (contains BAR0 for 8 VFs) > [ 2.736477] pci 0000:07:00.1: VF(n) BAR3 space: [mem > 0x00000000-0x0001ffff 64bit] (contains BAR3 for 8 VFs) > [ 2.763896] pci 0000:09:00.0: VF(n) BAR0 space: [mem > 0x00000000-0x0001ffff 64bit] (contains BAR0 for 8 VFs) > [ 2.774036] pci 0000:09:00.0: VF(n) BAR3 space: [mem > 0x00000000-0x0001ffff 64bit] (contains BAR3 for 8 VFs) > [ 2.784487] pci 0000:09:00.1: VF(n) BAR0 space: [mem > 0x00000000-0x0001ffff 64bit] (contains BAR0 for 8 VFs) > [ 2.794621] pci 0000:09:00.1: VF(n) BAR3 space: [mem > 0x00000000-0x0001ffff 64bit] (contains BAR3 for 8 VFs) > > I get this message and see the VF's in lspci but there isn't any IOMMU > group allocated. > [ 116.175246] igb 0000:07:00.0: 1 VFs allocated > [ 420.514477] igb 0000:09:00.1: 1 VFs allocated Hmm, I don't understand why they wouldn't have an iommu group associated with them. The quad-port 82576 cards had some special kind of brokenness about them though that I can't recall, perhaps something about ARI. Having no iommu group would imply that the devices don't live downstream of an iommu. Is this an Intel system? The DMAR ACPI table on Intel has path structures designed to take bus re-numbering into account, but maybe you're not on Intel or maybe the BIOS has done something particularly awful to negate this. > Would disabling devices in the BIOS help? Probably not. Logs please. dmesg, sudo lspci -vvv, /tmp/DMAR.dsl after running: # iasl -d -p /tmp/DMAR /sys/firmware/acpi/tables/DMAR (assuming an Intel system). Also: # find /sys/class/iommu/*/ -type l And # find /sys/kernel/iommu_groups/ -type l Thanks, Alex _______________________________________________ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
