Hi Tiago, thanks for trying it. The key should be resolving the issues shown in the serial console. Qemu should output the pseudo terminal devices, which it is attaching to. (/dev/pts/*) You have to use screen (Gentoo package and command line tool) to attach to them. Check the ubuntu wiki link, I sent, for an example. When you find the errors, check the archives for them. I think Alex has explained all kinds of errors a dozen times, so you might be able to resolve them with the existing hints. If you don‘t find any like yours, post them. It usually comes down to a missing Uefi Bios (in GPU). If you don’t find any errors, you probably have to tell the EFI console to boot. Google for it. I vaguely remember it being something like bootx64
Best regards, Manuel >Hello Manuel, > >Thank you very much for your help! You've definitely put me on the >right track. >Using the following qemu command I'm able to launch the vm without >errors. I will now work around getting the remainder of the hardware >working! > >LC_ALL=C /usr/bin/qemu-system-x86_64 \ > -machine >pc-i440fx-2.10,accel=kvm,usb=off,vmport=off,dump-guest-core=off,kernel_irqchip=on >\ >-cpu >host,migratable=no,+invtsc,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff >\ > -m 4096 \ > -realtime mlock=off \ > -smp 3,sockets=1,cores=3,threads=1 \ > -object iothread,id=iothread1 \ > -display none \ > -no-user-config \ > -nodefaults \ > -rtc base=localtime,clock=vm \ > -global kvm-pit.lost_tick_policy=delay \ > -no-hpet \ > -no-reboot \ > -serial pty \ > -serial pty \ > -monitor stdio \ > -nographic \ > -global PIIX4_PM.disable_s3=1 \ > -global PIIX4_PM.disable_s4=1 \ > -boot menu=off,strict=on \ > -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 \ > -device >ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 >\ > -device >ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 \ > -device >ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 \ > -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x7 \ >-drive >file=/usr/share/edk2-ovmf/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on >\ >-drive >file=/var/lib/libvirt/qemu/nvram/win10_VARS.fd,if=pflash,format=raw,unit=1 >\ > -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x6 \ > -device virtio-net-pci,netdev=hostnet0,addr=0xc \ > -netdev tap,ifname=tap0,script=no,downscript=no,id=hostnet0 \ > -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x2 \ > -device >scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 >\ >-drive >file=/home/tse/kvm_vm/test1-clone.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0 >\ > -device vfio-pci,host=01:00.1,id=hostdev1,bus=pci.0,addr=0x9 \ > -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0xa \ > -device usb-audio,id=sound0 \ > -msg timestamp=on 2>&1 > >On Thu, Jan 4, 2018 at 11:08 PM, Manuel Ullmann <la...@posteo.de> >wrote: >> Hm, >> >> I happen to use a qemu command line currently. I recall, that >guest/host >> sound sharing was close to impossible otherwise (using Alsa). >> >> I incorporated your drives and vfio devices, but since I don’t have >> discrete graphics, I have no idea how to get console output with >> OVMF. Found something here and added the serial ports and stdio >monitor >> as suggested. https://wiki.ubuntu.com/UEFI/OVMF >> You can attach to them with screen. >> >> If you don’t get any OVMF errors anymore, but still no output, you >could >> try to chmod the vfio device to 0666 temporarily, although this >should >> not be necessary, since you don’t get a permission denied error >anymore. >> >> I use the following to initialize my tap device. You probably don’t >use >> shorewall, so I commented the restart. Also you should replace the >> device placeholders and those from ip and gateway. Use ip link and ip >> addr to get yours. >> >> #!/bin/sh >> date >> /var/log/virsh.log >> sysctl net.ipv4.ip_forward=1 >> /var/log/virsh.log >> ip link add name br0 type bridge >>/var/log/virsh.log 2>&1 >> ip link set dev br0 up >>/var/log/virsh.log 2>&1 >> ip link set dev <your_dev> up >>/var/log/virsh.log 2>&1 >> ip link set dev <your_dev> master br0 >>/var/log/virsh.log 2>&1 >> ip tuntap add dev tap0 mode tap user qemu group qemu >>>/var/log/virsh.log 2>&1 >> ip link set dev tap0 up >>/var/log/virsh.log 2>&1 >> ip link set dev tap0 master br0 >>/var/log/virsh.log 2>&1 >> dhclient -4 br0 >>/var/log/virsh.log 2>&1 >> ip route change <your_ip>/24 via <gateway_ip> >>/var/log/virsh.log >2>&1 >> #rc-service shorewall restart >>/var/log/virsh.log 2>&1 >> #rc-service shorewall6 restart >>/var/log/virsh.log 2>&1 >> iptables --table nat --append POSTROUTING --out-interface br0 -j >MASQUERADE >>/var/log/virsh.log 2>&1 >> iptables --insert FORWARD --in-interface tap0 -j ACCEPT >>>/var/log/virsh.log 2>&1 >> echo "finished net setup" >>/var/log/virsh.log >> >> That VM won’t have input devices momentarily. Once qemu indicates a >> successful start and you get output, kill the VM and replace the >> commented -device usb-host entries with appropriate ids for your >> keyboard and mouse device. You get these with lsusb. Bus from lsusb >> output is hostbus and Device is hostaddr. Should the VM get stuck, >you >> can reattach the keyboard to regain control on the host. >> >> You were correct. The openrc runlevel is optional. If you would have >one >> and can write openrc-run scripts, you could put the preparation and >> cleaning up stuff into one, so that you later just have to do >something >> like init qemuvm and init default to switch into a VM friendly mode >> (less stuff running) and back. >> >https://wiki.gentoo.org/wiki/Handbook:AMD64/Working/Initscripts#Writing_initscripts >> >> Let me know, what you get. >> >> LC_ALL=C /usr/bin/qemu-system-x86_64 \ >> -machine >pc-i440fx-2.7,accel=kvm,usb=off,vmport=off,dump-guest-core=off,kernel_irqchip=on >\ >> -cpu >host,migratable=no,+invtsc,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff >\ >> -m 5000 \ >> -realtime mlock=off \ >> -smp 3,sockets=1,cores=3,threads=1 \ >> -object iothread,id=iothread1 \ >> -display none \ >> -no-user-config \ >> -nodefaults \ >> -rtc base=localtime,clock=vm \ >> -global kvm-pit.lost_tick_policy=delay \ >> -no-hpet \ >> -no-reboot \ >> -serial pty \ >> -serial pty \ >> -monitor stdio \ >> -nographic \ >> -global PIIX4_PM.disable_s3=1 \ >> -global PIIX4_PM.disable_s4=1 \ >> -boot menu=off,strict=on \ >> -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 \ >> -device >ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 >\ >> -device >ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 \ >> -device >ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 \ >> -device ahci,id=sata0,bus=pci.0,addr=0x8 \ >> -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 >\ >> -drive if=none,media=cdrom,id=drive-sata0-0-1,readonly=on \ >> -device ide-cd,bus=sata0.1,drive=drive-sata0-0-1,id=sata0-0-1 \ >> -device ide-cd,bus=sata0.4,drive=drive-sata0-0-4,id=sata0-0-4 \ >> -drive >file=/usr/share/edk2-ovmf/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on >\ >> -drive >file=/var/lib/libvirt/qemu/nvram/win10_VARS.fd,if=pflash,format=raw,unit=1 >\ >> -device >virtio-blk-pci,iothread=iothread1,scsi=off,bus=pci.0,addr=0xa,drive=drive-virtio-disk1,id=virtio-disk1 >\ >> -device >virtio-blk-pci,iothread=iothread1,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk4,id=virtio-disk4 >\ >> -device virtio-net-pci,netdev=hostnet0,addr=0xc \ >> -netdev >tap,ifname=tap0,script=no,downscript=no,vhost=on,id=hostnet0 \ >> -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x2 \ >> -device vfio-pci,host=01:00.1,id=hostdev1,bus=pci.0,addr=0x9 \ >> # -device >usb-host,hostbus=3,hostaddr=7,id=hostdev1,bus=usb.0,port=1 \ >> # -device >usb-host,hostbus=3,hostaddr=6,id=hostdev3,bus=usb.0,port=2 \ >> -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 \ >> -device usb-audio,id=sound0 \ >> -msg timestamp=on 2>&1 >> >>> Additionally, I experimented removing references to hostnet0, and >>> while I get no error messages, I also fail to get any image on the >>> display port in the monitor. >>> >>> /ts >>> >>> On Thu, Jan 4, 2018 at 7:28 PM, Tiago Seco <tiago.s...@gmail.com> >wrote: >>>> Hello Manuel >>>> >>>> With regards to permissions, I will try getting qemu to run with a >>>> lower privilege user, but since my setup is not working right now, >I >>>> want to impose as little restrictions as possible. Thank you for >the >>>> input, as well as the tip regarding hugepages. >>>> >>>> As far as your suggestion concerning openrc I must confess I do not >>>> fully follow what you're suggesting, but you made it sound like it >was >>>> an improvement rather than a requirement? >>>> >>>> Finally, for virsh start win10-clone >>>> /var/log/libvirt/qemu/win10-clone.log reads: >>>> >>>> 2018-01-04 20:10:41.397+0000: starting up libvirt version: 3.8.0, >qemu >>>> version: 2.10.1, hostname: rathalos >>>> LC_ALL=C >>>> >PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/usr/x86_64-pc-linux-gnu/gcc-bin/6.4.0:/usr/lib/llvm/4/bin:/opt/bin >>>> HOME=/root USER=root QEMU_AUDIO_DRV=none >/usr/bin/qemu-system-x86_64 >>>> -name guest=win10-clone,debug-threads=on -S -object >>>> >secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-8-win10-clone/master-key.aes >>>> -machine >pc-i440fx-2.10,accel=kvm,usb=off,vmport=off,dump-guest-core=off >>>> -cpu >>>> >Skylake-Client,ss=on,hypervisor=on,tsc_adjust=on,clflushopt=on,xsaves=on,pdpe1gb=on,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff >>>> -drive >file=/usr/share/edk2-ovmf/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on >>>> -drive >file=/var/lib/libvirt/qemu/nvram/win10_VARS.fd,if=pflash,format=raw,unit=1 >>>> -m 4096 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 >-uuid >>>> a673c9c8-a1f5-4c3d-97a7-16096bcf1b01 -display none -no-user-config >>>> -nodefaults -chardev >>>> >socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-8-win10-clone/monitor.sock,server,nowait >>>> -mon chardev=charmonitor,id=monitor,mode=control -rtc >>>> base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=delay >>>> -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global >>>> PIIX4_PM.disable_s4=1 -boot strict=on -device >>>> ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device >>>> >ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 >>>> -device >ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 >>>> -device >ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 >>>> -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x6 -device >>>> virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x7 -drive >>>> >file=/home/tse/kvm_vm/test1-clone.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0 >>>> -device >scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 >>>> -drive if=none,id=drive-ide0-0-1,readonly=on -device >>>> ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -drive >>>> if=none,id=drive-ide0-1-0,readonly=on -device >>>> ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev >>>> tap,fd=21,id=hostnet0 -device >>>> >rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:45:6f:82,bus=pci.0,addr=0x3 >>>> -chardev pty,id=charserial0 -device >>>> isa-serial,chardev=charserial0,id=serial0 -device >>>> AC97,id=sound0,bus=pci.0,addr=0x4 -chardev >>>> spicevmc,id=charredir0,name=usbredir -device >>>> usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 -chardev >>>> spicevmc,id=charredir1,name=usbredir -device >>>> usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 -device >>>> vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x2 -device >>>> vfio-pci,host=01:00.1,id=hostdev1,bus=pci.0,addr=0x9 -device >>>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on >>>> 2018-01-04T20:10:41.406580Z qemu-system-x86_64: -chardev >>>> pty,id=charserial0: char device redirected to /dev/pts/3 (label >>>> charserial0) >>>> 2018-01-04T20:10:41.432099Z qemu-system-x86_64: -device >>>> vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x2: vfio error: >>>> 0000:01:00.0: failed to open /dev/vfio/1: Permission denied >>>> 2018-01-04 20:10:41.442+0000: shutting down, reason=failed >>>> >>>> Interestingly enough, if I try to run that same command (as root) I >get: >>>> >>>> 2018-01-04T20:14:51.989313Z qemu-system-x86_64: -object >>>> >secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-8-win10-clone/master-key.aes: >>>> Unable to read >/var/lib/libvirt/qemu/domain-8-win10-clone/master-key.aes: >>>> Failed to open file >>>> '/var/lib/libvirt/qemu/domain-8-win10-clone/master-key.aes': No >such >>>> file or directory >>>> >>>> Removing '-object >>>> >secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-8-win10-clone/master-key.aes' >>>> I get: >>>> >>>> 2018-01-04T20:15:57.204616Z qemu-system-x86_64: -chardev >>>> >socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-8-win10-clone/monitor.sock,server,nowait: >>>> Failed to bind socket to >>>> /var/lib/libvirt/qemu/domain-8-win10-clone/monitor.sock: No such >file >>>> or directory >>>> >>>> I'm unsure wether or not this file should exist, if I create it, >and >>>> run the same command I get: >>>> >>>> 2018-01-04T20:17:07.490052Z qemu-system-x86_64: -chardev >>>> pty,id=charserial0: char device redirected to /dev/pts/3 (label >>>> charserial0) >>>> 2018-01-04T20:17:07.490500Z qemu-system-x86_64: -netdev >>>> tap,fd=21,id=hostnet0: TUNGETIFF ioctl() failed: Bad file >descriptor >>>> TUNSETOFFLOAD ioctl() failed: Bad file descriptor >>>> >>>> (the logs show nothing) >>>> >>>> If I do not create the file, and run the command without any >>>> references to the monitor (so without '-chardev >>>> >socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-8-win10-clone/monitor.sock,server,nowait >>>> -mon chardev=charmonitor,id=monitor,mode=control'), I get: >>>> >>>> 2018-01-04T20:22:04.201180Z qemu-system-x86_64: -chardev >>>> pty,id=charserial0: char device redirected to /dev/pts/3 (label >>>> charserial0) >>>> 2018-01-04T20:22:04.201597Z qemu-system-x86_64: -netdev >>>> tap,fd=21,id=hostnet0: TUNGETIFF ioctl() failed: Bad file >descriptor >>>> TUNSETOFFLOAD ioctl() failed: Bad file descriptor >>>> >>>> (Which leaves me wondering wether or not the -chardev and -mon >>>> switches are really necessary). >>>> >>>> I have very little experience with qemu, and could not spot >anything >>>> blatantly wrong in that command line. >>>> >>>> Once again, thank you very much for your help. >>>> >>>> /ts >>>> >>>> On Thu, Jan 4, 2018 at 5:32 PM, Manuel Ullmann <la...@posteo.de> >wrote: >>>>>> Hello, >>>>>> >>>>>> I had initially posted this on the gentoo forums, but since the >thread >>>>>> is getting little traction, perhaps here is a better place to ask >the >>>>>> question: >>>>>> >>>>>> I'm trying to achieve PCI-express GPU passthrough to a Window 10 >VM, >>>>>> however, when trying to start the virtual machine (as root) with >the >>>>>> hardware attached, this is what I get: >>>>>> >>>>>> virsh # start win10-clone >>>>>> error: Failed to start domain win10-clone >>>>>> error: internal error: process exited while connecting to >monitor: >>>>>> 2018-01-03T16:58:47.761436Z qemu-system-x86_64: -chardev >>>>>> pty,id=charserial0: char device redirected to /dev/pts/4 (label >>>>>> charserial0) >>>>>> 2018-01-03T16:58:47.786124Z qemu-system-x86_64: -device >>>>>> vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x2: vfio error: >>>>>> 0000:01:00.0: failed to open /dev/vfio/1: Permission denied >>>>>> >>>>>> The file however, exists: >>>>>> >>>>>> # file /dev/vfio/1 >>>>>> /dev/vfio/1: character special (250/0) >>>>>> # ls -l /dev/vfio/1 >>>>>> crw------- 1 root root 250, 0 Jan 3 16:51 /dev/vfio/1 >>>>>> >>>>>> >>>>>> I followed the steps detailed in >>>>>> https://wiki.installgentoo.com/index.php/PCI_passthrough for host >>>>>> preparation, and in >>>>>> >https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Setting_up_the_guest_OS >>>>>> for setting up the guest. >>>>>> >>>>>> Before running virsh, I: >>>>>> >>>>>> /etc/init.d/libvirtd stop >>>>>> /etc/init.d/libvirt-guests stop >>>>>> ./vfio-bind 0000:01:00.0 0000:01:00.1 >>>>>> /etc/init.d/libvirtd start >>>>>> /etc/init.d/libvirt-guests start >>>>>> >>>>>> >>>>>> I've tried playing around with /etc/libvirt/qemu.conf, namely >with the >>>>>> "group", "user", and "dynamic_ownership" settings - even though >I'm >>>>>> running everything as root - and with "cgroup_device_acl" (by >adding >>>>>> "/dev/vfio/1" to the list). >>>>>> I cannot figure out what the problem in my setup is or what to >try >>>>>> next. Is there anyone that can shine some light on this? >>>>>> >>>>>> >>>>>> System information, scripts used, and qemu USE flags can be found >>>>>> here: https://pastebin.com/XuGtDmSm >>>>>> >>>>>> Thank you! >>>>> Hello Tiago, >>>>> >>>>> I can vaguely recall having the same problem initially. Can you >verify, >>>>> that qemu really runs as root? If you strip the libvirt stuff from >the >>>>> qemu command line indicated in libvirtd logs and run it as root, >does >>>>> that work? >>>>> >>>>> Although I never would question the freedom, that Gentoo gives to >its >>>>> users, I still would suggest setting user and group to qemu >though. root >>>>> is quite insecure and we have a longstanding Intel CPU Bug >disclosed in >>>>> a few weeks tops, leaking page table content to userspace, which >affects >>>>> VM sandboxing gravely. Add kvm and audio groups to qemu user. If >you use >>>>> hugepages, those also have to belong to qemu. >>>>> >>>>> vfio nodes (or at least /dev/vfio/vfio) have not much for udev to >match >>>>> against, so it could be easier to modify permission in your >startup >>>>> scripts. I have this conveniently tackled by using a second >bootentry >>>>> with another default boot runlevel, which my /etc/local.d/*.start >>>>> scripts will recognize and prepare everything for the VM (with >openrc of >>>>> course). This makes less sense with discrete graphics passthrough >>>>> though, so I still would create a dedicated runlevel, but add a >>>>> openrc-run script, which prepares it, adding a rc_need entry in >>>>> conf.d/libvirtd. >>>>> >>>>> Best regards, >>>>> Manuel _______________________________________________ vfio-users mailing list vfio-users@redhat.com https://www.redhat.com/mailman/listinfo/vfio-users
