As of updating to kernel 4.5.4-1-vfio ( from 4.4.5-1-vfio ) am having a
strange issue in qemu (2.6.0) with my USB controller pass-through only
working once per host boot. Also interesting to note that my Nvidia HDMI
audio only works when the USB controller is not. The Nvidia HDMI audio
never worked at all until the USB controller pass-through stopped working.
Another change that happened at the same time as the kernel is I did a
fresh install of my Windows 10 guest to current stable. Was some random
insider build before that.
On VM shutdown I always get the following one or more times:
*qemu-system-x86_64: vfio: Cannot reset device 0000:00:14.0, no available
reset mechanism.*
*Steps to Reproduce:*
1. boot host system
2. start qemu
- USB host controller works here, HDMI audio does not
3. shut down and start VM again
- HDMI audio pass-through works now, USB host controller does not
All boots after this reflect state of step 6 until host system is rebooted
*Pass-through PCI devices:*
00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-H USB 3.0
xHCI Controller [8086:a12f] (rev 31) (prog-if 30 [XHCI])
Subsystem: ASUSTeK Computer Inc. Device [1043:8694]
Flags: bus master, medium devsel, latency 0, IRQ 171
Memory at 2fff010000 (64-bit, non-prefetchable) [size=64K]
Capabilities: [70] Power Management version 2
Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+
Kernel driver in use: vfio-pci
Kernel modules: xhci_pci
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM200 [GeForce
GTX 980 Ti] [10de:17c8] (rev a1) (prog-if 00 [VGA controller])
Subsystem: eVga.com. Corp. Device [3842:4995]
Flags: bus master, fast devsel, latency 0, IRQ 170
Memory at db000000 (32-bit, non-prefetchable) [size=16M]
Memory at 90000000 (64-bit, prefetchable) [size=256M]
Memory at a0000000 (64-bit, prefetchable) [size=32M]
I/O ports at e000 [size=128]
Expansion ROM at dc000000 [disabled] [size=512K]
Capabilities: [60] Power Management version 3
Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [78] Express Legacy Endpoint, MSI 00
Capabilities: [100] Virtual Channel
Capabilities: [258] L1 PM Substates
Capabilities: [128] Power Budgeting <?>
Capabilities: [420] Advanced Error Reporting
Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
Capabilities: [900] #19
Kernel driver in use: vfio-pci
Kernel modules: nouveau
01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:0fb0] (rev a1)
Subsystem: eVga.com. Corp. Device [3842:4995]
Flags: bus master, fast devsel, latency 0, IRQ 17
Memory at dc080000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [60] Power Management version 3
Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [78] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel
*Kernel CLI:*
initrd=\initramfs-linux-vfio.img
cryptdevice=UUID=bc838115-107d-41ef-a076-6a59a2a6bb43:cryptroot
root=/dev/mapper/cryptroot pcie_acs_override=downstream intel_iommu=on rw
*IOMMU status:*
[ 0.000000] Warning: PCIe ACS overrides enabled; This may allow
non-IOMMU protected peer-to-peer DMA
[ 0.000000] ACPI: DMAR 0x00000000365D85A8 0000B8 (v01 INTEL SKL
00000001 INTL 00000001)
[ 0.000000] DMAR: IOMMU enabled
[ 0.074566] DMAR: Host address width 39
[ 0.074568] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[ 0.074573] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap
1c0000c40660462 ecap 7e3ff0505e
[ 0.074574] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[ 0.074577] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap
d2008c40660462 ecap f050da
[ 0.074577] DMAR: RMRR base: 0x0000003630a000 end: 0x00000036329fff
[ 0.074578] DMAR: RMRR base: 0x00000038800000 end: 0x0000003affffff
[ 0.074580] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1
[ 0.074581] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[ 0.074581] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out
bit.
[ 0.074582] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the
BIOS setting.
[ 0.075960] DMAR-IR: Enabled IRQ remapping in xapic mode
[ 0.683801] DMAR: [Firmware Bug]: RMRR entry for device 04:00.0 is
broken - applying workaround
[ 0.683804] DMAR: [Firmware Bug]: RMRR entry for device 0b:00.0 is
broken - applying workaround
[ 0.683806] DMAR: No ATSR found
[ 0.683862] DMAR: dmar0: Using Queued invalidation
[ 0.684069] DMAR: dmar1: Using Queued invalidation
[ 0.684073] DMAR: Setting RMRR:
[ 0.684099] DMAR: Setting identity map for device 0000:00:02.0
[0x38800000 - 0x3affffff]
[ 0.684107] DMAR: Setting identity map for device 0000:00:14.0
[0x3630a000 - 0x36329fff]
[ 0.684115] DMAR: Setting identity map for device 0000:04:00.0
[0x3630a000 - 0x36329fff]
[ 0.684121] DMAR: Setting identity map for device 0000:0b:00.0
[0x3630a000 - 0x36329fff]
[ 0.684125] DMAR: Prepare 0-16MiB unity mapping for LPC
[ 0.684128] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 -
0xffffff]
[ 0.684155] DMAR: Intel(R) Virtualization Technology for Directed I/O
[ 88.848284] AMD IOMMUv2 driver by Joerg Roedel <jroe...@suse.de>
[ 88.848285] AMD IOMMUv2 functionality not available on this system
[111338.012019] DMAR: DRHD: handling fault status reg 2
[111338.012025] DMAR: DMAR:[DMA Read] Request device [00:14.0] fault addr
12acc0000
DMAR:[fault reason 02] Present bit in context entry is clear
*IOMMU groups:*
IOMMU group 0
00:00.0 Host bridge [0600]: Intel Corporation Skylake Host Bridge/DRAM
Registers [8086:191f] (rev 07)
IOMMU group 1
00:01.0 PCI bridge [0604]: Intel Corporation Skylake PCIe Controller (x16)
[8086:1901] (rev 07)
IOMMU group 2
00:01.1 PCI bridge [0604]: Intel Corporation Skylake PCIe Controller (x8)
[8086:1905] (rev 07)
IOMMU group 3
00:02.0 VGA compatible controller [0300]: Intel Corporation Skylake
Integrated Graphics [8086:1912] (rev 06)
IOMMU group 4
00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-H USB 3.0
xHCI Controller [8086:a12f] (rev 31)
IOMMU group 5
00:16.0 Communication controller [0780]: Intel Corporation Sunrise Point-H
CSME HECI #1 [8086:a13a] (rev 31)
IOMMU group 6
00:17.0 SATA controller [0106]: Intel Corporation Sunrise Point-H SATA
controller [AHCI mode] [8086:a102] (rev 31)
IOMMU group 7
00:1b.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Root Port
#17 [8086:a167] (rev f1)
IOMMU group 8
00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express
Root Port #1 [8086:a110] (rev f1)
00:1c.2 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express
Root Port #3 [8086:a112] (rev f1)
00:1c.4 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express
Root Port #5 [8086:a114] (rev f1)
04:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1142 USB 3.1 Host
Controller [1b21:1242]
05:00.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1187]
06:01.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1187]
06:02.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1187]
06:03.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1187]
06:04.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1187]
06:05.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1187]
06:06.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1187]
06:07.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1187]
07:00.0 USB controller [0c03]: VIA Technologies, Inc. VL805 USB 3.0 Host
Controller [1106:3483] (rev 01)
08:00.0 Network controller [0280]: Broadcom Corporation BCM4360 802.11ac
Wireless Network Adapter [14e4:43a0] (rev 03)
09:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA
Controller [1b21:0612] (rev 02)
0b:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042A USB 3.0
Host Controller [1b21:1142]
0e:00.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3 Bridge
[Alpine Ridge 4C 2015] [8086:1578]
0f:00.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3 Bridge
[Alpine Ridge 4C 2015] [8086:1578]
0f:01.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3 Bridge
[Alpine Ridge 4C 2015] [8086:1578]
0f:02.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3 Bridge
[Alpine Ridge 4C 2015] [8086:1578]
0f:04.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3 Bridge
[Alpine Ridge 4C 2015] [8086:1578]
44:00.0 USB controller [0c03]: Intel Corporation DSL6540 USB 3.1 Controller
[Alpine Ridge] [8086:15b6]
IOMMU group 9
00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express
Root Port #9 [8086:a118] (rev f1)
79:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd
NVMe SSD Controller [144d:a802] (rev 01)
IOMMU group 10
00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point-H LPC Controller
[8086:a145] (rev 31)
00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-H PMC
[8086:a121] (rev 31)
00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-H HD Audio
[8086:a170] (rev 31)
00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-H SMBus [8086:a123]
(rev 31)
IOMMU group 11
00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection
(2) I219-V [8086:15b8] (rev 31)
IOMMU group 12
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM200 [GeForce
GTX 980 Ti] [10de:17c8] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:0fb0] (rev a1)
IOMMU group 13
02:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc.
[AMD/ATI] Caicos PRO [Radeon HD 7450] [1002:677b]
02:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Caicos
HDMI Audio [Radeon HD 6400 Series] [1002:aa98]
*Hardware:*
H/W path Device Class Description
=================================================================
system System Product Name
(SKU)
/0 bus MAXIMUS VIII EXTREME
/0/0 memory 64KiB BIOS
/0/42 memory 128KiB L1 cache
/0/43 memory 128KiB L1 cache
/0/44 memory 1MiB L2 cache
/0/45 memory 8MiB L3 cache
/0/46 processor Core i7 (To Be Filled
By O.E.M.)
/0/47 memory 32GiB System Memory
/0/47/0 memory 16GiB DIMM
Synchronous 2133 MHz (0.5 ns)
/0/47/1 memory [empty]
/0/47/2 memory 16GiB DIMM
Synchronous 2133 MHz (0.5 ns)
/0/47/3 memory [empty]
/0/100 bridge Skylake Host
Bridge/DRAM Registers
/0/100/1 bridge Skylake PCIe
Controller (x16)
/0/100/1/0 display GM200 [GeForce GTX
980 Ti]
/0/100/1/0.1 multimedia NVIDIA Corporation
/0/100/1.1 bridge Skylake PCIe
Controller (x8)
/0/100/1.1/0 display Caicos PRO [Radeon HD
7450]
/0/100/1.1/0.1 multimedia Caicos HDMI Audio
[Radeon HD 6400 Series]
/0/100/2 display Skylake Integrated
Graphics
/0/100/14 bus Sunrise Point-H USB
3.0 xHCI Controller
/0/100/16 communication Sunrise Point-H CSME
HECI #1
/0/100/17 storage Sunrise Point-H SATA
controller [AHCI mode]
/0/100/1b bridge Sunrise Point-H PCI
Root Port #17
/0/100/1c bridge Sunrise Point-H PCI
Express Root Port #1
/0/100/1c/0 bus ASM1142 USB 3.1 Host
Controller
/0/100/1c/0/0 usb2 bus xHCI Host Controller
/0/100/1c/0/0/1 storage CDDVDW SU-208DB
/0/100/1c/0/1 usb1 bus xHCI Host Controller
/0/100/1c/0/1/2 multimedia HD Pro Webcam C920
/0/100/1c.2 bridge Sunrise Point-H PCI
Express Root Port #3
/0/100/1c.2/0 bridge ASMedia Technology
Inc.
/0/100/1c.2/0/1 bridge ASMedia Technology
Inc.
/0/100/1c.2/0/1/0 bus VL805 USB 3.0 Host
Controller
/0/100/1c.2/0/1/0/0 usb4 bus xHCI Host Controller
/0/100/1c.2/0/1/0/1 usb3 bus xHCI Host Controller
/0/100/1c.2/0/1/0/1/1 bus USB2.0 Hub
/0/100/1c.2/0/1/0/1/1/4 generic FT240X USB FIFO
/0/100/1c.2/0/2 bridge ASMedia Technology
Inc.
/0/100/1c.2/0/2/0 network BCM4360 802.11ac
Wireless Network Adapter
/0/100/1c.2/0/3 bridge ASMedia Technology
Inc.
/0/100/1c.2/0/3/0 storage ASM1062 Serial ATA
Controller
/0/100/1c.2/0/4 bridge ASMedia Technology
Inc.
/0/100/1c.2/0/5 bridge ASMedia Technology
Inc.
/0/100/1c.2/0/5/0 bus ASM1042A USB 3.0 Host
Controller
/0/100/1c.2/0/5/0/0 usb6 bus xHCI Host Controller
/0/100/1c.2/0/5/0/0/2 bus USB3.0 Hub
/0/100/1c.2/0/5/0/0/2/4 bus USB3.0 Hub
/0/100/1c.2/0/5/0/1 usb5 bus xHCI Host Controller
/0/100/1c.2/0/5/0/1/1 generic USB 10/100 LAN
/0/100/1c.2/0/5/0/1/2 bus USB2.0 Hub
/0/100/1c.2/0/5/0/1/2/1 bus USB2.0 Hub
/0/100/1c.2/0/5/0/1/2/4 bus USB2.0 Hub
/0/100/1c.2/0/5/0/1/2/4/4 input Yubikey 4 OTP+U2F+CCID
/0/100/1c.2/0/6 bridge ASMedia Technology
Inc.
/0/100/1c.2/0/7 bridge ASMedia Technology
Inc.
/0/100/1c.4 bridge Sunrise Point-H PCI
Express Root Port #5
/0/100/1c.4/0 bridge DSL6540 Thunderbolt 3
Bridge [Alpine Ridge 4C 2015]
/0/100/1c.4/0/0 bridge DSL6540 Thunderbolt 3
Bridge [Alpine Ridge 4C 2015]
/0/100/1c.4/0/1 bridge DSL6540 Thunderbolt 3
Bridge [Alpine Ridge 4C 2015]
/0/100/1c.4/0/2 bridge DSL6540 Thunderbolt 3
Bridge [Alpine Ridge 4C 2015]
/0/100/1c.4/0/2/0 bus DSL6540 USB 3.1
Controller [Alpine Ridge]
/0/100/1c.4/0/2/0/0 usb8 bus xHCI Host Controller
/0/100/1c.4/0/2/0/1 usb7 bus xHCI Host Controller
/0/100/1c.4/0/2/0/1/2 bus USB 2.0 Hub
/0/100/1c.4/0/2/0/1/2/2 input CST Laser Trackball
/0/100/1c.4/0/2/0/1/2/3 input ErgoDox ergonomic
keyboard
/0/100/1c.4/0/4 bridge DSL6540 Thunderbolt 3
Bridge [Alpine Ridge 4C 2015]
/0/100/1d bridge Sunrise Point-H PCI
Express Root Port #9
/0/100/1d/0 storage NVMe SSD Controller
/0/100/1f bridge Sunrise Point-H LPC
Controller
/0/100/1f.2 memory Memory controller
/0/100/1f.3 multimedia Sunrise Point-H HD
Audio
/0/100/1f.4 bus Sunrise Point-H SMBus
/0/100/1f.6 enp0s31f6 network Ethernet Connection
(2) I219-V
/1 power To Be Filled By O.E.M.
/2 scsi8 storage
/3 virbr0-nic network Ethernet interface
/4 enp11s0u1 network Ethernet interface
*Qemu CLI:*
QEMU_AUDIO_DRV=pa \
QEMU_PA_SAMPLES=4096 \
QEMU_PA_SERVER=localhost \
PULSE_SERVER=localhost \
qemu-system-x86_64 \
-enable-kvm \
-machine type=q35,accel=kvm \
-cpu host,kvm=off \
-smp 4,sockets=1,cores=2,threads=2 \
-m 8G \
-mem-path /dev/hugepages \
-device virtio-scsi-pci,id=scsi \
-drive file=/dev/sda,id=disk,format=raw,if=none \
-device scsi-hd,drive=disk \
-device vfio-pci,host=01:00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1 \
-drive
if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd
\
-drive if=pflash,format=raw,file=/tmp/tmp.CpwZ4KB6PL \
-soundhw hda \
-usb -device vfio-pci,host=00:14.0 \
-rtc clock=host,base=utc
--
Lance R. Vick
_______________________________________________
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users
