Hi,
Your nested virtualization may not support all features required for nested
passthrough.
Best regards
Corvin
This email contains confidential information. If you have received it in error,
you must not read, use, copy or pass on this e-mail or its attachments.
If you have received the e-mail in error, please inform me immediately by reply
e-mail and then delete this e-mail from your system. Thank you
Diese E-Mail enthaelt vertrauliche Informationen. Sollten Sie sie irrtuemlich
erhalten haben, duerfen Sie diese E-Mail oder ihre Anhaenge nicht lesen,
verwenden, kopieren oder weitergeben.
Sollten Sie die Mail versehentlich erhalten haben, teilen Sie mir dies bitte
umgehend per Antwort-E-Mail mit und loeschen Sie diese E-Mail dann aus Ihrem
System. Vielen Dank
Beckhoff Automation GmbH & Co. KG | Managing Director: Dipl. Phys. Hans Beckhoff
Registered office: Verl, Germany | Register court: Guetersloh HRA 7075
From: owner-freebsd-virtualizat...@freebsd.org
<owner-freebsd-virtualizat...@freebsd.org> On Behalf Of Mario Marietto
Sent: Tuesday, September 27, 2022 2:21 PM
To: FreeBSD virtualization <freebsd-virtualizat...@freebsd.org>
Subject: My nvidia gpu can't be passed through from Linux (qemu+kvm) to the 2.
guest OS (Puppy Linux) that I tried to virtualize on the 1. (FreeBSD 13.1)
CAUTION: External Email!!
Hello.
I've configured Xubuntu 22.04 so that it can allow me to pass thru my nvidia
gpu on the guest os vms. This time I've chosen freebsd 13.1 as a guest,because
I was curious to see if bhyve supports the passthrough of my nvidia gpu within
another guest os (Puppy Linux) that I have virtualized with bhyve. This is how
I have configured the FreeBSD 13.1 guest vm :
/boot/loader.conf
vmm_load="YES"
nmdm_load="YES"
if_tap_load="YES"
if_bridge_load="YES"
bridgestp_load="YES"
kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
kern.racct.enable=1
aio_load="YES"
cryptodev_load="YES"
zfs_load="YES"
verbose_loading="YES"
pptdevs="8/0/0 9/0/0 10/0/0 11/0/0"
pciconf -vl says that pptdevs reserved correctly the addresses 8/0/0 9/0/0
10/0/0 11/0/0 which belong to the nvidia GPU :
ppt0@pci0:8:0:0: class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de
device=0x1e04 subvendor=0x19da subdevice=0x2503
vendor = 'NVIDIA Corporation'
device = 'TU102 [GeForce RTX 2080 Ti]'
class = display
subclass = VGA
ppt1@pci0:9:0:0: class=0x040300 rev=0xa1 hdr=0x00 vendor=0x10de
device=0x10f7 subvendor=0x19da subdevice=0x2503
vendor = 'NVIDIA Corporation'
device = 'TU102 High Definition Audio Controller'
class = multimedia
subclass = HDA
ppt2@pci0:10:0:0: class=0x0c0330 rev=0xa1 hdr=0x00 vendor=0x10de
device=0x1ad6 subvendor=0x19da subdevice=0x2503
vendor = 'NVIDIA Corporation'
device = 'TU102 USB 3.1 Host Controller'
class = serial bus
subclass = USB
ppt3@pci0:11:0:0: class=0x0c8000 rev=0xa1 hdr=0x00 vendor=0x10de
device=0x1ad7 subvendor=0x19da subdevice=0x2503
vendor = 'NVIDIA Corporation'
device = 'TU102 USB Type-C UCSI Controller'
At this point,inside FreeBSD 13.1 guest OS,I tried to virtualize another OS,the
puppy Linux :
bhyve -S -c sockets=1,cores=1,threads=1 -m 2G -w -H -A \
-s 0,hostbridge \
-s 1,ahci-cd,/home/marietto/Desktop/bhyve/Files/fossapup64-9.5.iso,bootindex=1 \
-s 2,virtio-blk,/home/marietto/Desktop/bhyve/Files/puppy.img,bootindex=2 \
-s 8:0,passthru,8/0/0 \
-s 8:1,passthru,9/0/0 \
-s 8:2,passthru,10/0/0 \
-s 8:3,passthru,11/0/0 \
-s 10,virtio-net,tap18 \
-s 11,virtio-9p,sharename=/ \
-s 29,fbuf,tcp=0.0.0.0:5918<http://0.0.0.0:5918>,w=800,h=600,wait \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm0:18 < /dev/null & sleep 2 && vncviewer 0:18
unfortunately it gives this error :
bhyve: PCI device at 8/0/0 is not using the ppt(4) driverdevice emulation
initialization error: Device busy
I tried to remove the slots related to the passthru of the gpu and puppy booted
like a charm,like this one :
bhyve -S -c sockets=1,cores=1,threads=1 -m 2G -w -H -A \
-s 0,hostbridge \
-s 1,ahci-cd,/home/marietto/Desktop/bhyve/Files/fossapup64-9.5.iso,bootindex=1 \
-s 2,virtio-blk,/home/marietto/Desktop/bhyve/Files/puppy.img,bootindex=2 \
-s 10,virtio-net,tap18 \
-s 11,virtio-9p,sharename=/ \
-s 29,fbuf,tcp=0.0.0.0:5918<http://0.0.0.0:5918>,w=800,h=600,wait \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm0:18 < /dev/null & sleep 2 && vncviewer 0:18
So,where could be the error in this specific scenario ? Nested vm works,but I
can't pass through the gpu,even if the host OS (xubuntu) makes it available to
the guest.
--
Mario.
