Hello Manuel,
Thank you very much for your help! You've definitely put me on the right track.
Using the following qemu command I'm able to launch the vm without
errors. I will now work around getting the remainder of the hardware
working!
LC_ALL=C /usr/bin/qemu-system-x86_64 \
-machine
pc-i440fx-2.10,accel=kvm,usb=off,vmport=off,dump-guest-core=off,kernel_irqchip=on
\
-cpu
host,migratable=no,+invtsc,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff
\
-m 4096 \
-realtime mlock=off \
-smp 3,sockets=1,cores=3,threads=1 \
-object iothread,id=iothread1 \
-display none \
-no-user-config \
-nodefaults \
-rtc base=localtime,clock=vm \
-global kvm-pit.lost_tick_policy=delay \
-no-hpet \
-no-reboot \
-serial pty \
-serial pty \
-monitor stdio \
-nographic \
-global PIIX4_PM.disable_s3=1 \
-global PIIX4_PM.disable_s4=1 \
-boot menu=off,strict=on \
-device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 \
-device
ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5
\
-device
ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 \
-device
ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x7 \
-drive
file=/usr/share/edk2-ovmf/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on
\
-drive
file=/var/lib/libvirt/qemu/nvram/win10_VARS.fd,if=pflash,format=raw,unit=1
\
-device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x6 \
-device virtio-net-pci,netdev=hostnet0,addr=0xc \
-netdev tap,ifname=tap0,script=no,downscript=no,id=hostnet0 \
-device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x2 \
-device
scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1
\
-drive
file=/home/tse/kvm_vm/test1-clone.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0
\
-device vfio-pci,host=01:00.1,id=hostdev1,bus=pci.0,addr=0x9 \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0xa \
-device usb-audio,id=sound0 \
-msg timestamp=on 2>&1
On Thu, Jan 4, 2018 at 11:08 PM, Manuel Ullmann <la...@posteo.de> wrote:
> Hm,
>
> I happen to use a qemu command line currently. I recall, that guest/host
> sound sharing was close to impossible otherwise (using Alsa).
>
> I incorporated your drives and vfio devices, but since I don’t have
> discrete graphics, I have no idea how to get console output with
> OVMF. Found something here and added the serial ports and stdio monitor
> as suggested. https://wiki.ubuntu.com/UEFI/OVMF
> You can attach to them with screen.
>
> If you don’t get any OVMF errors anymore, but still no output, you could
> try to chmod the vfio device to 0666 temporarily, although this should
> not be necessary, since you don’t get a permission denied error anymore.
>
> I use the following to initialize my tap device. You probably don’t use
> shorewall, so I commented the restart. Also you should replace the
> device placeholders and those from ip and gateway. Use ip link and ip
> addr to get yours.
>
> #!/bin/sh
> date >> /var/log/virsh.log
> sysctl net.ipv4.ip_forward=1 >> /var/log/virsh.log
> ip link add name br0 type bridge >>/var/log/virsh.log 2>&1
> ip link set dev br0 up >>/var/log/virsh.log 2>&1
> ip link set dev <your_dev> up >>/var/log/virsh.log 2>&1
> ip link set dev <your_dev> master br0 >>/var/log/virsh.log 2>&1
> ip tuntap add dev tap0 mode tap user qemu group qemu >>/var/log/virsh.log 2>&1
> ip link set dev tap0 up >>/var/log/virsh.log 2>&1
> ip link set dev tap0 master br0 >>/var/log/virsh.log 2>&1
> dhclient -4 br0 >>/var/log/virsh.log 2>&1
> ip route change <your_ip>/24 via <gateway_ip> >>/var/log/virsh.log 2>&1
> #rc-service shorewall restart >>/var/log/virsh.log 2>&1
> #rc-service shorewall6 restart >>/var/log/virsh.log 2>&1
> iptables --table nat --append POSTROUTING --out-interface br0 -j MASQUERADE
> >>/var/log/virsh.log 2>&1
> iptables --insert FORWARD --in-interface tap0 -j ACCEPT >>/var/log/virsh.log
> 2>&1
> echo "finished net setup" >>/var/log/virsh.log
>
> That VM won’t have input devices momentarily. Once qemu indicates a
> successful start and you get output, kill the VM and replace the
> commented -device usb-host entries with appropriate ids for your
> keyboard and mouse device. You get these with lsusb. Bus from lsusb
> output is hostbus and Device is hostaddr. Should the VM get stuck, you
> can reattach the keyboard to regain control on the host.
>
> You were correct. The openrc runlevel is optional. If you would have one
> and can write openrc-run scripts, you could put the preparation and
> cleaning up stuff into one, so that you later just have to do something
> like init qemuvm and init default to switch into a VM friendly mode
> (less stuff running) and back.
> https://wiki.gentoo.org/wiki/Handbook:AMD64/Working/Initscripts#Writing_initscripts
>
> Let me know, what you get.
>
> LC_ALL=C /usr/bin/qemu-system-x86_64 \
> -machine
> pc-i440fx-2.7,accel=kvm,usb=off,vmport=off,dump-guest-core=off,kernel_irqchip=on
> \
> -cpu
> host,migratable=no,+invtsc,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff \
> -m 5000 \
> -realtime mlock=off \
> -smp 3,sockets=1,cores=3,threads=1 \
> -object iothread,id=iothread1 \
> -display none \
> -no-user-config \
> -nodefaults \
> -rtc base=localtime,clock=vm \
> -global kvm-pit.lost_tick_policy=delay \
> -no-hpet \
> -no-reboot \
> -serial pty \
> -serial pty \
> -monitor stdio \
> -nographic \
> -global PIIX4_PM.disable_s3=1 \
> -global PIIX4_PM.disable_s4=1 \
> -boot menu=off,strict=on \
> -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 \
> -device
> ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6
> \
> -device
> ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 \
> -device
> ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 \
> -device ahci,id=sata0,bus=pci.0,addr=0x8 \
> -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 \
> -drive if=none,media=cdrom,id=drive-sata0-0-1,readonly=on \
> -device ide-cd,bus=sata0.1,drive=drive-sata0-0-1,id=sata0-0-1 \
> -device ide-cd,bus=sata0.4,drive=drive-sata0-0-4,id=sata0-0-4 \
> -drive
> file=/usr/share/edk2-ovmf/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on
> \
> -drive
> file=/var/lib/libvirt/qemu/nvram/win10_VARS.fd,if=pflash,format=raw,unit=1 \
> -device
> virtio-blk-pci,iothread=iothread1,scsi=off,bus=pci.0,addr=0xa,drive=drive-virtio-disk1,id=virtio-disk1
> \
> -device
> virtio-blk-pci,iothread=iothread1,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk4,id=virtio-disk4
> \
> -device virtio-net-pci,netdev=hostnet0,addr=0xc \
> -netdev tap,ifname=tap0,script=no,downscript=no,vhost=on,id=hostnet0 \
> -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x2 \
> -device vfio-pci,host=01:00.1,id=hostdev1,bus=pci.0,addr=0x9 \
> # -device usb-host,hostbus=3,hostaddr=7,id=hostdev1,bus=usb.0,port=1 \
> # -device usb-host,hostbus=3,hostaddr=6,id=hostdev3,bus=usb.0,port=2 \
> -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 \
> -device usb-audio,id=sound0 \
> -msg timestamp=on 2>&1
>
>> Additionally, I experimented removing references to hostnet0, and
>> while I get no error messages, I also fail to get any image on the
>> display port in the monitor.
>>
>> /ts
>>
>> On Thu, Jan 4, 2018 at 7:28 PM, Tiago Seco <tiago.s...@gmail.com> wrote:
>>> Hello Manuel
>>>
>>> With regards to permissions, I will try getting qemu to run with a
>>> lower privilege user, but since my setup is not working right now, I
>>> want to impose as little restrictions as possible. Thank you for the
>>> input, as well as the tip regarding hugepages.
>>>
>>> As far as your suggestion concerning openrc I must confess I do not
>>> fully follow what you're suggesting, but you made it sound like it was
>>> an improvement rather than a requirement?
>>>
>>> Finally, for virsh start win10-clone
>>> /var/log/libvirt/qemu/win10-clone.log reads:
>>>
>>> 2018-01-04 20:10:41.397+0000: starting up libvirt version: 3.8.0, qemu
>>> version: 2.10.1, hostname: rathalos
>>> LC_ALL=C
>>> PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/usr/x86_64-pc-linux-gnu/gcc-bin/6.4.0:/usr/lib/llvm/4/bin:/opt/bin
>>> HOME=/root USER=root QEMU_AUDIO_DRV=none /usr/bin/qemu-system-x86_64
>>> -name guest=win10-clone,debug-threads=on -S -object
>>> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-8-win10-clone/master-key.aes
>>> -machine pc-i440fx-2.10,accel=kvm,usb=off,vmport=off,dump-guest-core=off
>>> -cpu
>>> Skylake-Client,ss=on,hypervisor=on,tsc_adjust=on,clflushopt=on,xsaves=on,pdpe1gb=on,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff
>>> -drive
>>> file=/usr/share/edk2-ovmf/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on
>>> -drive
>>> file=/var/lib/libvirt/qemu/nvram/win10_VARS.fd,if=pflash,format=raw,unit=1
>>> -m 4096 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid
>>> a673c9c8-a1f5-4c3d-97a7-16096bcf1b01 -display none -no-user-config
>>> -nodefaults -chardev
>>> socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-8-win10-clone/monitor.sock,server,nowait
>>> -mon chardev=charmonitor,id=monitor,mode=control -rtc
>>> base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=delay
>>> -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global
>>> PIIX4_PM.disable_s4=1 -boot strict=on -device
>>> ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device
>>> ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5
>>> -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1
>>> -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2
>>> -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x6 -device
>>> virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x7 -drive
>>> file=/home/tse/kvm_vm/test1-clone.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0
>>> -device
>>> scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1
>>> -drive if=none,id=drive-ide0-0-1,readonly=on -device
>>> ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -drive
>>> if=none,id=drive-ide0-1-0,readonly=on -device
>>> ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev
>>> tap,fd=21,id=hostnet0 -device
>>> rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:45:6f:82,bus=pci.0,addr=0x3
>>> -chardev pty,id=charserial0 -device
>>> isa-serial,chardev=charserial0,id=serial0 -device
>>> AC97,id=sound0,bus=pci.0,addr=0x4 -chardev
>>> spicevmc,id=charredir0,name=usbredir -device
>>> usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 -chardev
>>> spicevmc,id=charredir1,name=usbredir -device
>>> usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 -device
>>> vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x2 -device
>>> vfio-pci,host=01:00.1,id=hostdev1,bus=pci.0,addr=0x9 -device
>>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on
>>> 2018-01-04T20:10:41.406580Z qemu-system-x86_64: -chardev
>>> pty,id=charserial0: char device redirected to /dev/pts/3 (label
>>> charserial0)
>>> 2018-01-04T20:10:41.432099Z qemu-system-x86_64: -device
>>> vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x2: vfio error:
>>> 0000:01:00.0: failed to open /dev/vfio/1: Permission denied
>>> 2018-01-04 20:10:41.442+0000: shutting down, reason=failed
>>>
>>> Interestingly enough, if I try to run that same command (as root) I get:
>>>
>>> 2018-01-04T20:14:51.989313Z qemu-system-x86_64: -object
>>> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-8-win10-clone/master-key.aes:
>>> Unable to read /var/lib/libvirt/qemu/domain-8-win10-clone/master-key.aes:
>>> Failed to open file
>>> '/var/lib/libvirt/qemu/domain-8-win10-clone/master-key.aes': No such
>>> file or directory
>>>
>>> Removing '-object
>>> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-8-win10-clone/master-key.aes'
>>> I get:
>>>
>>> 2018-01-04T20:15:57.204616Z qemu-system-x86_64: -chardev
>>> socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-8-win10-clone/monitor.sock,server,nowait:
>>> Failed to bind socket to
>>> /var/lib/libvirt/qemu/domain-8-win10-clone/monitor.sock: No such file
>>> or directory
>>>
>>> I'm unsure wether or not this file should exist, if I create it, and
>>> run the same command I get:
>>>
>>> 2018-01-04T20:17:07.490052Z qemu-system-x86_64: -chardev
>>> pty,id=charserial0: char device redirected to /dev/pts/3 (label
>>> charserial0)
>>> 2018-01-04T20:17:07.490500Z qemu-system-x86_64: -netdev
>>> tap,fd=21,id=hostnet0: TUNGETIFF ioctl() failed: Bad file descriptor
>>> TUNSETOFFLOAD ioctl() failed: Bad file descriptor
>>>
>>> (the logs show nothing)
>>>
>>> If I do not create the file, and run the command without any
>>> references to the monitor (so without '-chardev
>>> socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-8-win10-clone/monitor.sock,server,nowait
>>> -mon chardev=charmonitor,id=monitor,mode=control'), I get:
>>>
>>> 2018-01-04T20:22:04.201180Z qemu-system-x86_64: -chardev
>>> pty,id=charserial0: char device redirected to /dev/pts/3 (label
>>> charserial0)
>>> 2018-01-04T20:22:04.201597Z qemu-system-x86_64: -netdev
>>> tap,fd=21,id=hostnet0: TUNGETIFF ioctl() failed: Bad file descriptor
>>> TUNSETOFFLOAD ioctl() failed: Bad file descriptor
>>>
>>> (Which leaves me wondering wether or not the -chardev and -mon
>>> switches are really necessary).
>>>
>>> I have very little experience with qemu, and could not spot anything
>>> blatantly wrong in that command line.
>>>
>>> Once again, thank you very much for your help.
>>>
>>> /ts
>>>
>>> On Thu, Jan 4, 2018 at 5:32 PM, Manuel Ullmann <la...@posteo.de> wrote:
>>>>> Hello,
>>>>>
>>>>> I had initially posted this on the gentoo forums, but since the thread
>>>>> is getting little traction, perhaps here is a better place to ask the
>>>>> question:
>>>>>
>>>>> I'm trying to achieve PCI-express GPU passthrough to a Window 10 VM,
>>>>> however, when trying to start the virtual machine (as root) with the
>>>>> hardware attached, this is what I get:
>>>>>
>>>>> virsh # start win10-clone
>>>>> error: Failed to start domain win10-clone
>>>>> error: internal error: process exited while connecting to monitor:
>>>>> 2018-01-03T16:58:47.761436Z qemu-system-x86_64: -chardev
>>>>> pty,id=charserial0: char device redirected to /dev/pts/4 (label
>>>>> charserial0)
>>>>> 2018-01-03T16:58:47.786124Z qemu-system-x86_64: -device
>>>>> vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x2: vfio error:
>>>>> 0000:01:00.0: failed to open /dev/vfio/1: Permission denied
>>>>>
>>>>> The file however, exists:
>>>>>
>>>>> # file /dev/vfio/1
>>>>> /dev/vfio/1: character special (250/0)
>>>>> # ls -l /dev/vfio/1
>>>>> crw------- 1 root root 250, 0 Jan 3 16:51 /dev/vfio/1
>>>>>
>>>>>
>>>>> I followed the steps detailed in
>>>>> https://wiki.installgentoo.com/index.php/PCI_passthrough for host
>>>>> preparation, and in
>>>>> https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Setting_up_the_guest_OS
>>>>> for setting up the guest.
>>>>>
>>>>> Before running virsh, I:
>>>>>
>>>>> /etc/init.d/libvirtd stop
>>>>> /etc/init.d/libvirt-guests stop
>>>>> ./vfio-bind 0000:01:00.0 0000:01:00.1
>>>>> /etc/init.d/libvirtd start
>>>>> /etc/init.d/libvirt-guests start
>>>>>
>>>>>
>>>>> I've tried playing around with /etc/libvirt/qemu.conf, namely with the
>>>>> "group", "user", and "dynamic_ownership" settings - even though I'm
>>>>> running everything as root - and with "cgroup_device_acl" (by adding
>>>>> "/dev/vfio/1" to the list).
>>>>> I cannot figure out what the problem in my setup is or what to try
>>>>> next. Is there anyone that can shine some light on this?
>>>>>
>>>>>
>>>>> System information, scripts used, and qemu USE flags can be found
>>>>> here: https://pastebin.com/XuGtDmSm
>>>>>
>>>>> Thank you!
>>>> Hello Tiago,
>>>>
>>>> I can vaguely recall having the same problem initially. Can you verify,
>>>> that qemu really runs as root? If you strip the libvirt stuff from the
>>>> qemu command line indicated in libvirtd logs and run it as root, does
>>>> that work?
>>>>
>>>> Although I never would question the freedom, that Gentoo gives to its
>>>> users, I still would suggest setting user and group to qemu though. root
>>>> is quite insecure and we have a longstanding Intel CPU Bug disclosed in
>>>> a few weeks tops, leaking page table content to userspace, which affects
>>>> VM sandboxing gravely. Add kvm and audio groups to qemu user. If you use
>>>> hugepages, those also have to belong to qemu.
>>>>
>>>> vfio nodes (or at least /dev/vfio/vfio) have not much for udev to match
>>>> against, so it could be easier to modify permission in your startup
>>>> scripts. I have this conveniently tackled by using a second bootentry
>>>> with another default boot runlevel, which my /etc/local.d/*.start
>>>> scripts will recognize and prepare everything for the VM (with openrc of
>>>> course). This makes less sense with discrete graphics passthrough
>>>> though, so I still would create a dedicated runlevel, but add a
>>>> openrc-run script, which prepares it, adding a rc_need entry in
>>>> conf.d/libvirtd.
>>>>
>>>> Best regards,
>>>> Manuel
_______________________________________________
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users
