I bet you've already tried this but looking through your xml file for the guest I see a few inconsistencies when compared against the suggestions on the Arch Wiki [here](https://wiki.archlinux.org/index.php/PCI_passthrough_ via_OVMF#.22Error_43:_Driver_failed_to_load.22_on_Nvidia_ GPUs_passed_to_Windows_VMs).
I found the important bits to the <hyperv> tags. Hope this helps. Alex On 3 January 2018 at 23:28, Robert Dahlstrom <robert.j.dahlst...@gmail.com> wrote: > Hello, I have been trying to setup a Windows 10 guest on my Linux host, > passing through an nvidia 1060. I've been having a very difficult time, > getting the dreaded code 43. I haven't followed an individual guide, but > have taken bits from https://wiki.archlinux.org/ind > ex.php/PCI_passthrough_via_OVMF and Alex Williamson's when relevant among > others. > > I have tried just about every possible solution I've found, but have come > up short. > > One thing I'll point out right away: in my VM, I put two virtual disks. On > one I installed windows, on the other I installed Debian 9. When booting > into the linux guest OS, with the proprietary nvidia driver, **everything > works**. When I boot into windows: code 43. Based on that, I think I have > most of my stuff correctly setup, and I am just at the receiving end of > nvidia treachery. I hope I'm wrong about that though. > > Since both cards are nvidia cards, I use vfio.conf to prevent the nvidia > driver from loading on the guest. The guest GPU is in an IOMMU group that > consists solely of it, its HDMI audio device, and the Host Bridge/PCI > Bridge devices. Both the video and audio devices are forwarded to the guest. > > What I have tried: > > 1) Seemingly every permutation of hidden kvm, vendor_id settings, hyperv > settings, both i440fx and q35, BIOS and OVMF, > > 2) My current situation has my guest GPU in the first PCIe slot, with my > host GPU in the third. I think the first 2 slots share an IOMMU group, > presumably for SLI or some such. This puts my guest GPU as the boot card, > which is what the system BIOS and bootloader comes up on. I read that may > cause trouble, and so I've tried swapping the cards. This I don't believe > works because there is a boatload of other stuff in the IOMMU group of the > third slot (Group 0), such as my host SATA and USB controllers, that I > can't forward. > > 3) Passing disable_vga in vfio.conf, doesn't seem to do anything > > 4) Setting MSI interrupt signaling in windows for both the video and > audio devices. > > What I have not tried: > > 1) Switching which GPU I use for host and guest. If I can't use the 1060 > the whole thing is pointless. It may tell me if the boot GPU issue is the > cause of my woe, however. > > 2) I have read sometimes booting with the virtual graphics device (Spice > + QXL) causes problems. Trying that, I have not been able to boot up at all > without it. Presence of -nographics on qemu commandline seems to make no > difference. When I blast the display from the config, the qemu process > starts up and sits at 0% and never does anything, and I can't later remote > desktop or ssh into the guest. > > 3) I have not yet successfully gone through the process of personally > downloading the ROM from the card and using it. I have instead grabbed a > ROM file from techpowerup.com, and verified that it has both BIOS and EFI > portions. I can passthrough in linux regardless whether I use the ROM or > not, and in Windows it makes no difference, code 43 for both. When I try to > download the ROM myself I get the Invalid PCI ROM header signature error, > and an input/output error. > > 4) Giving up and buying an AMD card. > > Any ideas what I'm missing? The oldest drivers from NVidia are from march > 2017, most info I can find on this subject is from 2016 or earlier. Do the > later drivers have additional checks that qemu is not able to fool at this > time? Thanks for any help, and if you need any additional information I'll > be happy to provide it. > > System information: > > * Linux 4.9.0-3-amd64 (Debian 9) > * qemu-x86_64 version 2.8.1(Debian 1:2.8+dfsg-6+deb9u3) > * virsh -version: 3.0.0 > * Processor: AMD Ryzen 7 1700X > * Host GPU: NVidia 740, evga > * Guest GPU: Nvidia 1060, Asus > * Guest NVidia Drivers: 378.78 - the oldest I can get off the website. > > /etc/modprobe.d/vfio.conf : > options vfio-pci ids=10de:1c02,10de:10f1 disable_vga=1 > > lspci -nnv: (This is after vfio is setup, vfio-pci drivers are in use as > expected) > > .... > 21:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK107 > [GeForce GT 740] [10de:0fc8] (rev a1) (prog-if 00 [VGA controller]) > Subsystem: eVga.com. Corp. GK107 [GeForce GT 740] [3842:3747] > Flags: bus master, fast devsel, latency 0, IRQ 62 > Memory at f4000000 (32-bit, non-prefetchable) [size=16M] > Memory at e0000000 (64-bit, prefetchable) [size=256M] > Memory at f0000000 (64-bit, prefetchable) [size=32M] > I/O ports at d000 [size=128] > [virtual] Expansion ROM at f5000000 [disabled] [size=512K] > Capabilities: <access denied> > Kernel driver in use: nvidia > Kernel modules: nvidia > > 21:00.1 Audio device [0403]: NVIDIA Corporation GK107 HDMI Audio > Controller [10de:0e1b] (rev a1) > Subsystem: eVga.com. Corp. GK107 HDMI Audio Controller [3842:3747] > Flags: bus master, fast devsel, latency 0, IRQ 43 > Memory at f5080000 (32-bit, non-prefetchable) [size=16K] > Capabilities: <access denied> > Kernel driver in use: snd_hda_intel > Kernel modules: snd_hda_intel > > .... > > 28:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP106 > [GeForce GTX 1060 3GB] [10de:1c02] (rev a1) (prog-if 00 [VGA controller]) > Subsystem: ASUSTeK Computer Inc. GP106 [GeForce GTX 1060 3GB] > [1043:85b1] > Flags: bus master, fast devsel, latency 0, IRQ 63 > Memory at f6000000 (32-bit, non-prefetchable) [size=16M] > Memory at c0000000 (64-bit, prefetchable) [size=256M] > Memory at d0000000 (64-bit, prefetchable) [size=32M] > I/O ports at e000 [size=128] > Expansion ROM at 000c0000 [disabled] [size=128K] > Capabilities: <access denied> > Kernel driver in use: vfio-pci > Kernel modules: nvidia > > 28:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10f1] (rev a1) > Subsystem: ASUSTeK Computer Inc. Device [1043:85b1] > Flags: bus master, fast devsel, latency 0, IRQ 65 > Memory at f7080000 (32-bit, non-prefetchable) [size=16K] > Capabilities: <access denied> > Kernel driver in use: vfio-pci > Kernel modules: snd_hda_intel > > .... > > > IOMMU groups: > > IOMMU Group 0 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1452] > IOMMU Group 0 00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1453] > IOMMU Group 0 00:01.3 PCI bridge [0604]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1453] > IOMMU Group 0 01:00.0 Non-Volatile memory controller [0108]: Samsung > Electronics Co Ltd Device [144d:a804] > IOMMU Group 0 03:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. > [AMD] Device [1022:43b9] (rev 02) > IOMMU Group 0 03:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. > [AMD] Device [1022:43b5] (rev 02) > IOMMU Group 0 03:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. > [AMD] Device [1022:43b0] (rev 02) > IOMMU Group 0 1d:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. > [AMD] Device [1022:43b4] (rev 02) > IOMMU Group 0 1d:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. > [AMD] Device [1022:43b4] (rev 02) > IOMMU Group 0 1d:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. > [AMD] Device [1022:43b4] (rev 02) > IOMMU Group 0 1d:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. > [AMD] Device [1022:43b4] (rev 02) > IOMMU Group 0 21:00.0 VGA compatible controller [0300]: NVIDIA Corporation > GK107 [GeForce GT 740] [10de:0fc8] (rev a1) > IOMMU Group 0 21:00.1 Audio device [0403]: NVIDIA Corporation GK107 HDMI > Audio Controller [10de:0e1b] (rev a1) > IOMMU Group 0 25:00.0 USB controller [0c03]: ASMedia Technology Inc. > Device [1b21:1343] > IOMMU Group 0 26:00.0 Ethernet controller [0200]: Intel Corporation I211 > Gigabit Network Connection [8086:1539] (rev 03) > IOMMU Group 1 00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1452] > IOMMU Group 2 00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1452] > IOMMU Group 2 00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1453] > IOMMU Group 2 28:00.0 VGA compatible controller [0300]: NVIDIA Corporation > GP106 [GeForce GTX 1060 3GB] [10de:1c02] (rev a1) > IOMMU Group 2 28:00.1 Audio device [0403]: NVIDIA Corporation Device > [10de:10f1] (rev a1) > IOMMU Group 3 00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1452] > IOMMU Group 4 00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1452] > IOMMU Group 4 00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1454] > IOMMU Group 4 29:00.0 Non-Essential Instrumentation [1300]: Advanced Micro > Devices, Inc. [AMD] Device [1022:145a] > IOMMU Group 4 29:00.2 Encryption controller [1080]: Advanced Micro > Devices, Inc. [AMD] Device [1022:1456] > IOMMU Group 4 29:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. > [AMD] Device [1022:145c] > IOMMU Group 5 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1452] > IOMMU Group 5 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1454] > IOMMU Group 5 2a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro > Devices, Inc. [AMD] Device [1022:1455] > IOMMU Group 5 2a:00.2 SATA controller [0106]: Advanced Micro Devices, Inc. > [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51) > IOMMU Group 5 2a:00.3 Audio device [0403]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1457] > IOMMU Group 6 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH > SMBus Controller [1022:790b] (rev 59) > IOMMU Group 6 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. > [AMD] FCH LPC Bridge [1022:790e] (rev 51) > IOMMU Group 7 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1460] > IOMMU Group 7 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1461] > IOMMU Group 7 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1462] > IOMMU Group 7 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1463] > IOMMU Group 7 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1464] > IOMMU Group 7 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1465] > IOMMU Group 7 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1466] > IOMMU Group 7 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. > [AMD] Device [1022:1467] > > ROM I got from techpowerup: > > rom-parser /opt/Asus.GTX1060.3072.161104.rom > Valid ROM signature found @a00h, PCIR offset 1a0h > PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 1c02, class: 030000 > PCIR: revision 0, vendor revision: 1 > Valid ROM signature found @fa00h, PCIR offset 1ch > PCIR: type 3 (EFI), vendor: 10de, device: 1c02, class: 030000 > PCIR: revision 3, vendor revision: 0 > EFI: Signature Valid, Subsystem: Boot, Machine: X64 > Last image > > libvirt config: > > <domain type='kvm'> > <name>winefitry3</name> > <uuid>69a4ea18-6a00-41f4-afbc-80fd42ad60fd</uuid> > <memory unit='KiB'>4194304</memory> > <currentMemory unit='KiB'>4194304</currentMemory> > <vcpu placement='static'>4</vcpu> > <os> > <type arch='x86_64' machine='pc-i440fx-2.8'>hvm</type> > <loader readonly='yes' type='pflash'>/usr/share/OVMF/ > OVMF_CODE.fd</loader> > <nvram>/var/lib/libvirt/qemu/nvram/winefitry3_VARS.fd</nvram> > <bootmenu enable='yes'/> > </os> > <features> > <acpi/> > <apic/> > <hyperv> > <relaxed state='on'/> > <vapic state='on'/> > <spinlocks state='on' retries='8191'/> > <vendor_id state='on' value='AMDMachine57'/> > </hyperv> > <kvm> > <hidden state='on'/> > </kvm> > <vmport state='off'/> > </features> > <cpu mode='host-passthrough'/> > <clock offset='utc'> > <timer name='rtc' tickpolicy='catchup'/> > <timer name='pit' tickpolicy='delay'/> > <timer name='hpet' present='no'/> > </clock> > <on_poweroff>destroy</on_poweroff> > <on_reboot>restart</on_reboot> > <on_crash>restart</on_crash> > <pm> > <suspend-to-mem enabled='no'/> > <suspend-to-disk enabled='no'/> > </pm> > <devices> > <emulator>/usr/bin/kvm</emulator> > <disk type='file' device='cdrom'> > <driver name='qemu' type='raw'/> > <source file='/opt/KNOPPIX_V7.7.1DVD-2016-10-22-EN.iso'/> > <target dev='sda' bus='sata'/> > <readonly/> > <address type='drive' controller='0' bus='0' target='0' unit='0'/> > </disk> > <disk type='file' device='disk'> > <driver name='qemu' type='vmdk'/> > <source file='/vmware/passthrough/winefi3.vmdk'/> > <target dev='sdb' bus='sata'/> > <address type='drive' controller='0' bus='0' target='0' unit='1'/> > </disk> > <disk type='file' device='disk'> > <driver name='qemu' type='vmdk'/> > <source file='/vmware/passthrough/linuxtest.vmdk'/> > <target dev='sdc' bus='sata'/> > <boot order='1'/> > <address type='drive' controller='0' bus='0' target='0' unit='2'/> > </disk> > <controller type='usb' index='0' model='ich9-ehci1'> > <address type='pci' domain='0x0000' bus='0x00' slot='0x05' > function='0x7'/> > </controller> > <controller type='usb' index='0' model='ich9-uhci1'> > <master startport='0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x05' > function='0x0' multifunction='on'/> > </controller> > <controller type='usb' index='0' model='ich9-uhci2'> > <master startport='2'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x05' > function='0x1'/> > </controller> > <controller type='usb' index='0' model='ich9-uhci3'> > <master startport='4'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x05' > function='0x2'/> > </controller> > <controller type='pci' index='0' model='pci-root'/> > <controller type='sata' index='0'> > <address type='pci' domain='0x0000' bus='0x00' slot='0x06' > function='0x0'/> > </controller> > <controller type='virtio-serial' index='0'> > <address type='pci' domain='0x0000' bus='0x00' slot='0x07' > function='0x0'/> > </controller> > <interface type='direct'> > <mac address='52:54:00:2c:a2:8e'/> > <source dev='enp38s0' mode='bridge'/> > <model type='virtio'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' > function='0x0'/> > </interface> > <serial type='pty'> > <target port='0'/> > </serial> > <console type='pty'> > <target type='serial' port='0'/> > </console> > <channel type='unix'> > <target type='virtio' name='org.qemu.guest_agent.0'/> > <address type='virtio-serial' controller='0' bus='0' port='1'/> > </channel> > <channel type='spicevmc'> > <target type='virtio' name='com.redhat.spice.0'/> > <address type='virtio-serial' controller='0' bus='0' port='2'/> > </channel> > <input type='tablet' bus='usb'> > <address type='usb' bus='0' port='1'/> > </input> > <input type='mouse' bus='ps2'/> > <input type='keyboard' bus='ps2'/> > <graphics type='spice' autoport='yes'> > <listen type='address'/> > </graphics> > <sound model='ich6'> > <address type='pci' domain='0x0000' bus='0x00' slot='0x04' > function='0x0'/> > </sound> > <video> > <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' > primary='yes'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x09' > function='0x0'/> > </video> > <hostdev mode='subsystem' type='pci' managed='yes'> > <source> > <address domain='0x0000' bus='0x28' slot='0x00' function='0x0'/> > </source> > <rom file='/opt/Asus.GTX1060.3072.161104.rom'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x08' > function='0x0'/> > </hostdev> > <hostdev mode='subsystem' type='pci' managed='yes'> > <source> > <address domain='0x0000' bus='0x28' slot='0x00' function='0x1'/> > </source> > <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' > function='0x0'/> > </hostdev> > <redirdev bus='usb' type='spicevmc'> > <address type='usb' bus='0' port='2'/> > </redirdev> > <redirdev bus='usb' type='spicevmc'> > <address type='usb' bus='0' port='3'/> > </redirdev> > <memballoon model='virtio'> > <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' > function='0x0'/> > </memballoon> > </devices> > </domain> > > Thanks for reading this far! > > _______________________________________________ > vfio-users mailing list > vfio-users@redhat.com > https://www.redhat.com/mailman/listinfo/vfio-users > > -- Alex Kretzschmar 07796 863 373
_______________________________________________ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
