Hi, If you using Radeon RX with i440fx type vm, GPU attached to pcie-root-complex trick is maybe needed.
i440fx and recent AMD Crimson drivers https://www.redhat.com/archives/vfio-users/2016-April/msg00153.html Remove your Radeon RX Vega 56 passthrough config from <hostdev>~~</hostdev>section and using libvirt "<qemu:arg>" option, some thing like this. <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,rombar=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=root.1,addr=00.1'/> </qemu:commandline> Or, Use Q35 type VM with Intel ioh3420 pcie-root-port. For example, see below. <controller type='pci' index='2' model='pcie-root-port'> <model name='ioh3420'/> <target chassis='2' port='0x11'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/> </controller> ~~~~~ snip ~~~~~~ <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0xaf' slot='0x00' function='0x0'/> </source> <rom bar='on'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0' multifunction='on'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0xaf' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x1'/> </hostdev> "model name=ioh3420" is Intel emulated pcie-root-port. defalut "model name=pcie-root-port" is QEMU Paravirtualized pcie-root-port. Radeon driver with "ioh3420" root port trick works fine on my workstation. (but... not RX Vega 56, I'm using RX 460.) Cheers, Hisato. > On Thu, 28 Mar 2019 at 20:32, Tmplt <tmplt dragons rocks> wrote: > > > > On 3/28/19 8:02 PM, Kash Pande wrote: > > > kvm module parameter ignore_msrs is needed here. > > Is this much different from the > > > > echo 1 > /sys/module/kvm/parameters/ignore_msrs > > I have a very similar setup with a Sapphire RX Vega 56 and also > observing this lockup. > I am using an unpatched Debian 4.19.0. Qemu 3.1.0. Libvirt 5.0.0. A > single host core is pegged at 100%. I've attached my XML too. > $ cat /sys/module/kvm/parameters/ignore_msrs > Y > I cannot find any logs indicating any problems... > $ sudo journalctl | grep kvm > Mar 28 20:35:52 socrates kernel: Command line: > BOOT_IMAGE=/boot/vmlinuz-4.19.0-4-amd64 > root=/dev/mapper/VolGroup00-lvdebian ro iommu=pt > amd_iommu=force_isolation rd.driver.pre=vfio-pci hugepagesz=1GB > hugepages=16 vfio_iommu_type1.allow_unsafe_interrupts=1 > kvm.ignore_msrs=1 > Mar 28 20:35:52 socrates kernel: Kernel command line: > BOOT_IMAGE=/boot/vmlinuz-4.19.0-4-amd64 > root=/dev/mapper/VolGroup00-lvdebian ro iommu=pt > amd_iommu=force_isolation rd.driver.pre=vfio-pci hugepagesz=1GB > hugepages=16 vfio_iommu_type1.allow_unsafe_interrupts=1 > kvm.ignore_msrs=1 > Mar 28 20:35:52 socrates kernel: kvm: Nested Virtualization enabled > Mar 28 20:35:52 socrates kernel: kvm: Nested Paging enabled > $ sudo journalctl | grep msrs > Mar 28 20:35:52 socrates kernel: Command line: > BOOT_IMAGE=/boot/vmlinuz-4.19.0-4-amd64 > root=/dev/mapper/VolGroup00-lvdebian ro iommu=pt > amd_iommu=force_isolation rd.driver.pre=vfio-pci hugepagesz=1GB > hugepages=16 vfio_iommu_type1.allow_unsafe_interrupts=1 > kvm.ignore_msrs=1 > Mar 28 20:35:52 socrates kernel: Kernel command line: > BOOT_IMAGE=/boot/vmlinuz-4.19.0-4-amd64 > root=/dev/mapper/VolGroup00-lvdebian ro iommu=pt > amd_iommu=force_isolation rd.driver.pre=vfio-pci hugepagesz=1GB > hugepages=16 vfio_iommu_type1.allow_unsafe_interrupts=1 > kvm.ignore_msrs=1 > > <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0 '> > <name>win10_backup</name> > <uuid>4b697e59-793e-450f-8722-38e785831796</uuid> > <title>Windows 10</title> > <description>Windows gaming VM</description> > <memory unit='KiB'>16777216</memory> > <currentMemory unit='KiB'>16777216</currentMemory> > <vcpu placement='static'>8</vcpu> > <iothreads>1</iothreads> > <cputune> > <vcpupin vcpu='0' cpuset='8'/> > <vcpupin vcpu='1' cpuset='9'/> > <vcpupin vcpu='2' cpuset='10'/> > <vcpupin vcpu='3' cpuset='11'/> > <vcpupin vcpu='4' cpuset='12'/> > <vcpupin vcpu='5' cpuset='13'/> > <vcpupin vcpu='6' cpuset='14'/> > <vcpupin vcpu='7' cpuset='15'/> > <emulatorpin cpuset='0-1'/> > <iothreadpin iothread='1' cpuset='0-1'/> > </cputune> > <os> > <type arch='x86_64' machine='pc-i440fx-3.1'>hvm</type> > <loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.fd</loader> > <nvram>/var/lib/libvirt/qemu/nvram/win10_backup_VARS.fd</nvram> > </os> > <features> > <acpi/> > <apic/> > <hyperv> > <relaxed state='on'/> > <vapic state='on'/> > <spinlocks state='on' retries='8191'/> > </hyperv> > <vmport state='off'/> > </features> > <cpu mode='host-passthrough' check='none'> > <topology sockets='1' cores='4' threads='2'/> > <cache level='3' mode='emulate'/> > <feature policy='require' name='topoext'/> > </cpu> > <clock offset='localtime'> > <timer name='rtc' tickpolicy='catchup'/> > <timer name='pit' tickpolicy='delay'/> > <timer name='hpet' present='no'/> > <timer name='hypervclock' present='yes'/> > </clock> > <on_poweroff>destroy</on_poweroff> > <on_reboot>restart</on_reboot> > <on_crash>destroy</on_crash> > <pm> > <suspend-to-mem enabled='no'/> > <suspend-to-disk enabled='no'/> > </pm> > <devices> > <emulator>/usr/bin/kvm</emulator> > <disk type='file' device='disk'> > <driver name='qemu' type='raw' io='threads'/> > <source file='/var/lib/libvirt/images/win10-clean_backup.raw'/> > <target dev='vda' bus='virtio'/> > <boot order='2'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> > </disk> > <disk type='file' device='cdrom'> > <driver name='qemu' type='raw'/> > <source file='/var/lib/libvirt/images/virtio-win-0.1.160.iso'/> > <target dev='sda' bus='sata'/> > <readonly/> > <address type='drive' controller='0' bus='0' target='0' unit='0'/> > </disk> > <controller type='pci' index='0' model='pci-root'/> > <controller type='usb' index='0' model='nec-xhci'> > <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> > </controller> > <controller type='sata' index='0'> > <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> > </controller> > <interface type='network'> > <mac address='52:54:00:39:d1:fc'/> > <source network='default'/> > <model type='virtio'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> > </interface> > <input type='mouse' bus='ps2'/> > <input type='keyboard' bus='ps2'/> > <sound model='ich6'> > <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> > </sound> > <hostdev mode='subsystem' type='pci' managed='yes'> > <source> > <address domain='0x0000' bus='0x13' slot='0x00' function='0x0'/> > </source> > <rom bar='off'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/> > </hostdev> > <hostdev mode='subsystem' type='usb' managed='yes'> > <source> > <vendor id='0x046d'/> > <product id='0xc24c'/> > <address bus='3' device='7'/> > </source> > <address type='usb' bus='0' port='1'/> > </hostdev> > <hostdev mode='subsystem' type='usb' managed='yes'> > <source> > <vendor id='0x046d'/> > <product id='0xc228'/> > <address bus='3' device='9'/> > </source> > <address type='usb' bus='0' port='2'/> > </hostdev> > <memballoon model='virtio'> > <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> > </memballoon> > </devices> > <qemu:commandline> > <qemu:env name='QEMU_AUDIO_DRV' value='pa'/> > <qemu:env name='QEMU_PA_SERVER' value='/run/user/1000/pulse/native'/> > </qemu:commandline> > </domain>
_______________________________________________ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
