Hi, I have a SuperMicro A2SDi-16C-HLN4F which uses recently released Denverton SoC (Intel Atom C3955). This mainboard has one PCI-E 3.0 x4 slot, but whatever card is included there it doesn't work with VFIO.
1. All tried cards work fine in another mainboard using VT-d and in another mainboard using AMD-IOMMU. 2. All tried cards report DPC events (AER fixed them). However using them on host seems to work fine (tried it for some time now) [29136.808030] dpc 0000:00:09.0:pcie010: DPC containment event, status:0x1f00 source:0x0000 [29136.808045] pcieport 0000:00:09.0: AER: Corrected error received: id=0048 [29136.808051] pcieport 0000:00:09.0: PCIe Bus Error: severity=Corrected, type=Data Link Layer, id=0048(Transmitter ID) [29136.809533] pcieport 0000:00:09.0: device [8086:19a4] error status/mask=00001000/00002000 [29136.811079] pcieport 0000:00:09.0: [12] Replay Timer Timeout 00:09.0 is the PCI bridge and current device behind it is a Digital Devices GmbH Octopus DVB Adapter. The above error is what I see on host if using device there, as soon as I start using it vie VFIO I get the following: Nov 17 05:06:13 server.theraso.int kernel: vfio-pci 0000:01:00.0: enabling device (0140 -> 0142) Nov 17 05:06:14 server.theraso.int kernel: vfio_bar_restore: 0000:01:00.0 reset recovery - restoring bars Nov 17 05:06:36 server.theraso.int kernel: vfio_bar_restore: 0000:01:00.0 reset recovery - restoring bars Nov 17 05:06:36 server.theraso.int kernel: vfio_bar_restore: 0000:01:00.0 reset recovery - restoring bars Inside VM I get immediate after boot: Nov 17 00:25:18 vdr.theraso.int kernel: Disabling IRQ #11 Nov 17 00:25:18 vdr.theraso.int kernel: [<ffffffffc074d060>] qxl_irq_handler [qxl] Nov 17 00:25:18 vdr.theraso.int kernel: [<ffffffffc03a4570>] usb_hcd_irq [usbcore] Nov 17 00:25:18 vdr.theraso.int kernel: handlers: Nov 17 00:25:18 vdr.theraso.int kernel: secondary_startup_64+0x9f/0x9f Nov 17 00:25:18 vdr.theraso.int kernel: x86_64_start_kernel+0x13e/0x161 Nov 17 00:25:18 vdr.theraso.int kernel: x86_64_start_reservations+0x24/0x26 Nov 17 00:25:18 vdr.theraso.int kernel: ? early_idt_handler_array+0x120/0x120 Nov 17 00:25:18 vdr.theraso.int kernel: start_kernel+0x496/0x4b7 Nov 17 00:25:18 vdr.theraso.int kernel: rest_init+0xd5/0xe0 Nov 17 00:25:18 vdr.theraso.int kernel: cpu_startup_entry+0x73/0x80 Nov 17 00:25:18 vdr.theraso.int kernel: do_idle+0x175/0x1e0 Nov 17 00:25:18 vdr.theraso.int kernel: default_idle_call+0x23/0x30 Nov 17 00:25:18 vdr.theraso.int kernel: arch_cpu_idle+0xf/0x20 Nov 17 00:25:18 vdr.theraso.int kernel: default_idle+0x20/0x130 ... Nov 17 00:25:18 vdr.theraso.int kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.11.0-20171110_100015-anatol 04/01/2014 Nov 17 00:25:18 vdr.theraso.int kernel: CPU: 0 PID: 0 Comm: swapper/0 Tainted: G C 4.13.5-1-ARCH #1 Nov 17 00:25:18 vdr.theraso.int kernel: irq 11: nobody cared (try booting with the "irqpoll" option) If I shutdown the VM, host puts device in a state which makes it not working anymore: Nov 17 05:14:00 server.theraso.int kernel: vfio-pci 0000:01:00.0: Failed to return from FLR Nov 17 05:13:58 server.theraso.int kernel: vfio-pci 0000:01:00.0: timed out waiting for pending transaction; performing function level reset anyway Next VM start: Nov 17 00:28:22 server.theraso.int kernel: vfio-pci 0000:01:00.0: Refused to change power state, currently in D3 Moreover I've tried this all already with a RealTek RTL-8169 NIC. The issue remains the same. As mentioned in the beginning the devices works fine on other boards. Any help would be much appreciated to narrow down the problem. The DPC events occurs also in case of not using VFIO at all. --Maik _______________________________________________ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
