On Thu, 23 Aug 2018 15:57:11 +0800 acewind <acew...@gmail.com> wrote:
> Yesterday I tried IGD with Ubuntu 18.04 but failed: > https://www.redhat.com/archives/vfio-users/2018-August/msg00026.html > > Today I use the same hardware enviroment, reinstall OS to CentOS 7.3 > without GUI, upgrade kernel and qemu version: > > CPU: i3-4010U, support vt-d. > > [root@test ~]# lspci | grep VGA > 00:02.0 VGA compatible controller: Intel Corporation Haswell-ULT Integrated > Graphics Controller (rev 09) > (Only one IGD with vga and hdmi interface) > > [root@test ~]# uname -r > 4.18.3-1.el7.centos.x86_64 > > [root@test ~]# /usr/libexec/qemu-kvm --version > QEMU emulator version 2.10.0(qemu-kvm-ev-2.10.0-21.el7_5.4.1) > Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers You might be going further off track with this, the official RH products do not support IGD assignment in any way other than kvmgt, ie. GVT-g, ie. vGPU. In fact the RHEL kernel specifically disables some of the IGD and VGA features necessary for legacy mode IGD assignment. I don't know how much of this Centos carries to their product, but if you're moving to Centos with the expectation that IGD assignment is better supported there from the RH lineage, it's not. > Then I blacklist i915, forbid *fb: > > [root@test ~]# cat /etc/default/grub | grep GRUB_CMDLINE_LINUX > GRUB_CMDLINE_LINUX="crashkernel=auto rhgb quiet intel_iommu=on iommu=pt > video=vesafb:off vga=normal nofb nomodeset i915.modeset=0 > nouveau.modeset=0 rd.driver.blacklist=nouveau,i915 plymouth.ignore-udev" > > [root@test ~]# cat /etc/modprobe.d/blacklist-i915.conf > install i915 /bin/true > > After rebooted: > [root@test ~]# cat /proc/cmdline > BOOT_IMAGE=/vmlinuz-4.18.3-1.el7.centos.x86_64 > root=UUID=78feccee-2f23-4ab6-b386-2b51d1664da7 ro crashkernel=auto rhgb > quiet intel_iommu=on iommu=pt video=vesafb:off vga=normal nofb nomodeset > i915.modeset=0 nouveau.modeset=0 rd.driver.blacklist=nouveau,i915 > plymouth.ignore-udev > > [root@test ~]# cat /proc/iomem | grep fb > 01c031d1-02463fbf : Kernel data > f8000000-fbffffff : PCI MMCONFIG 0000 [bus 00-3f] > f8000000-fbffffff : Reserved > f8000000-fbffffff : pnp 00:05 > > [root@test ~]# lsmod | grep i915 > ... nothing ... It's really not necessary to go to these sorts of lengths to disable i915. You can simply 'virsh nodedev-detach pci_0000_00_02_0'. IME, the place where i915 causes trouble is if you later attempt to re-bind the device to i915. > Now I begin to load vfio modules. > > [root@test ~]# modprobe vfio > [root@test ~]# modprobe vfio-pci > > [root@test ~]# echo "vfio-pci" > > /sys/bus/pci/devices/0000:00:02.0/driver_override > [root@test ~]# ./vfio-pci-bind.sh 0000:00:02.0 > (script ref: > https://github.com/andre-richter/vfio-pci-bind/blob/master/vfio-pci-bind.sh) > > [root@test ~]# lspci -v -s 00:02.0 > 00:02.0 VGA compatible controller: Intel Corporation Haswell-ULT Integrated > Graphics Controller (rev 09) (prog-if 00 [VGA controller]) > Subsystem: Intel Corporation Haswell-ULT Integrated Graphics Controller > Flags: fast devsel, IRQ 16 > Memory at a7c00000 (64-bit, non-prefetchable) [size=4M] > Memory at b0000000 (64-bit, prefetchable) [size=256M] > I/O ports at 4000 [size=64] > [virtual] Expansion ROM at 000c0000 [disabled] [size=128K] > Capabilities: [90] MSI: Enable- Count=1/1 Maskable- 64bit- > Capabilities: [d0] Power Management version 2 > Capabilities: [a4] PCI Advanced Features > Kernel driver in use: vfio-pci > Kernel modules: i915 > > Now virsh create a win7 vm, xml file is: > > <domain type='kvm' id='3' xmlns:qemu=' > http://libvirt.org/schemas/domain/qemu/1.0'> > <name>windows_7_ultimate_with_sp1_x64</name> > <uuid>2fcccb90-f622-481a-8819-367c642c0a09</uuid> > <memory unit='KiB'>2097152</memory> > <currentMemory unit='KiB'>2097152</currentMemory> > <vcpu placement='static'>2</vcpu> > <resource> > <partition>/machine</partition> > </resource> > <os> > <type arch='x86_64' machine='pc-i440fx-rhel7.5.0'>hvm</type> > <boot dev='hd'/> > </os> > <features> > <acpi/> > <apic/> > <hyperv> > <relaxed state='on'/> > <vapic state='on'/> > <spinlocks state='on' retries='8191'/> > </hyperv> > </features> > <cpu mode='custom' match='exact' check='full'> > <model fallback='forbid'>Haswell-noTSX</model> > <feature policy='require' name='vme'/> > <feature policy='require' name='f16c'/> > <feature policy='require' name='rdrand'/> > <feature policy='require' name='hypervisor'/> > <feature policy='require' name='arat'/> > <feature policy='require' name='xsaveopt'/> > <feature policy='require' name='abm'/> > </cpu> > <clock offset='localtime'> > <timer name='rtc' tickpolicy='catchup'/> > <timer name='pit' tickpolicy='delay'/> > <timer name='hpet' present='no'/> > <timer name='hypervclock' present='yes'/> > </clock> > <on_poweroff>destroy</on_poweroff> > <on_reboot>restart</on_reboot> > <on_crash>destroy</on_crash> > <pm> > <suspend-to-mem enabled='no'/> > <suspend-to-disk enabled='no'/> > </pm> > <devices> > <emulator>/usr/libexec/qemu-kvm</emulator> > <disk type='file' device='disk'> > <driver name='qemu' type='qcow2'/> > <source file='/home/windows_7_ultimate_with_sp1_x64.qcow2'/> > <backingStore/> > <target dev='hda' bus='ide'/> > <alias name='ide0-0-0'/> > <address type='drive' controller='0' bus='0' target='0' unit='0'/> > </disk> > > <hostdev mode='subsystem' type='pci' managed='yes'> > <driver name='vfio'/> > <source> > <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> > </source> > <alias name='hostdev0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x0'/> > </hostdev> > </devices> > > <qemu:commandline> > <qemu:arg value='-bios'/> > <qemu:arg value='/usr/share/seabios/bios.bin'/> > <qemu:arg value='-chardev'/> > <qemu:arg value='file,id=seabios,path=/tmp/bios.log'/> > <qemu:arg value='-device'/> > <qemu:arg value='isa-debugcon,iobase=0x402,chardev=seabios'/> > </qemu:commandline> > </domain> > > After created, the vm is running, monitor screen turn to black then no > output. > The end of seabios log is: > > ... > Scan for VGA option rom > Running option rom at c000:0003 > > Any problem with VGA rom? In the BIOS I have already enabled CSM and the > option rom of video is set to legacy. > > Ref to http://lists.gnu.org/archive/html/qemu-discuss/2018-04/msg00052.html > and https://github.com/awilliam/rom-parser > > I dump the vga rom: > > [root@test rom]# echo 1 > /sys/devices/pci0000:00/0000:00:02.0/rom > [root@test rom]# cat /sys/devices/pci0000:00/0000:00:02.0/rom > vbios.dump > [root@test rom]# echo 0 > /sys/devices/pci0000:00/0000:00:02.0/rom > > [root@test rom]# ./rom-parser/rom-parser vbios.dump > Valid ROM signature found @0h, PCIR offset 40h > PCIR: type 0 (x86 PC-AT), vendor: 8086, device: 0406, class: 030000 > PCIR: revision 3, vendor revision: 0 > Last image > > [root@test rom]# lspci -nns 00:02.0 > 00:02.0 VGA compatible controller [0300]: Intel Corporation Haswell-ULT > Integrated Graphics Controller [8086:0a16] (rev 09) > > The device code in the rom is diffrent from vga device? So I modify it with > rom-fixer command: > > [root@test rom]# ./rom-parser/rom-fixer ./vbios.dump > Valid ROM signature found @0h, PCIR offset 40h > PCIR: type 0 (x86 PC-AT), vendor: 8086, device: 0406, class: 030000 > PCIR: revision 3, vendor revision: 0 > > Modify vendor ID 8086? (y/n): y > New vendor ID: 8086 > Overwrite vendor ID with 8086? (y/n): y > Modify device ID 0406? (y/n): y > New device ID: 0a16 > Overwrite device ID with 0a16? (y/n): y > Last image > ROM checksum is invalid, fix? (y/n): y The only reason this should be necessary is if you're already using an externally provided rom via <rom>. QEMU will do the above for you automatically when it's reading the rom directly from the device. So there's no net effect of this change. > Then I edit the vm xml, and romfile args to hostdev: > <rom file='/path/to/vbios.dump'/> > > Then destroy and create the vm again, still black screen ... > > Is there any more methods to debug？ I'd use upstream kernel and QEMU. Your previous post indicated DMAR faults. A couple of those are normal, but you seem to have a fair number of them and the address is incrementing as if our attempt to trap and relocate stolen memory isn't working. It's possible this particular GPU uses a different page table format that we don't know how to handle yet. Thanks, Alex _______________________________________________ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
