[Bug 262936] bhyve randomly fails to boot/stop/reboot
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=262936 Bug ID: 262936 Summary: bhyve randomly fails to boot/stop/reboot Product: Base System Version: 13.0-STABLE Hardware: amd64 OS: Any Status: New Severity: Affects Only Me Priority: --- Component: bhyve Assignee: virtualization@FreeBSD.org Reporter: berger...@yahoo.co.uk THe system is 13.0-STABLE FreeBSD 13.0-STABLE #0 stable/13-n249491-be8e341777c (installed from one of the snapshots) My Windows 10 VM randomly fails to boot (bhyve console hangs dead), after which it cannot be started again without the machine reboot. I have passthrough network and USB cards, so may be related, maybe not. What happens in detail: SOMETIMES boot console hangs dead at UEFI booting stage, after which `bhyvectl --vm $my_vm --force-reset / --force-poweroff` commands don't work at all. Boot console itself hangs and so does the corresponding 'bhyve' process. The `bhyve --vm $myvm --destroy` command DOES something: ls /dev/vmm is empty, VM destroyed, but the `bhyve` process is still hanging there. It disappears upon user logout/login (desktop). But when, after these manipulations, I try to start the VM anew it gives off this error: bhyve: Failed to map MSI-X table BAR on 12/0/0: Device busy bhyve: failed to initialize BARs for PCI 12/0/0 device emulation initialization error: Device busy Here is my command: bhyve -S -c sockets=1,cores=4,threads=2 -m 8G -H -w \ -s 0,hostbridge \ -s 3,passthru,8/0/0 \ -s 4,virtio-blk,/bhyve/win-alt.img,sectorsize=512/4096 \ -s 6,hda,rec=/dev/dsp,play=/dev/dsp \ -s 8,passthru,12/0/0 \ -s 29,fbuf,tcp=0.0.0.0:5900,w=1600,h=900,wait \ -s 30,xhci,tablet \ -s 31,lpc \ -l com1,stdio \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ windows-10 Is there anything else that can be "reset" there without a complete machine reboot? -- You are receiving this mail because: You are the assignee for the bug.
[Bug 262936] bhyve randomly fails to boot/stop/reboot
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=262936 --- Comment #1 from berger...@yahoo.co.uk --- NOTE: this started happening only since I migrated to STABLE. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 260178] bhyve: passthru makes ahci-hd boot fail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260178 --- Comment #13 from commit-h...@freebsd.org --- A commit in branch releng/13.1 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=1c6abf864ecd3bbf07ace2018f9aab45b6406ce2 commit 1c6abf864ecd3bbf07ace2018f9aab45b6406ce2 Author: Bjoern A. Zeeb AuthorDate: 2022-03-18 20:39:06 + Commit: Bjoern A. Zeeb CommitDate: 2022-03-30 15:33:47 + bhyve: Do not remove guest physical addresses from IOMMU host domain This permits I/O devices on the host to directly access wired memory dedicated to guests using passthru devices. Note that wired memory belonging to guests that do not use passthru devices has always been accessible by I/O devices on the host. bhyve maps guest physical addresses into the user address space of the bhyve process by mmap'ing /dev/vmm/. Device models pass pointers derived from this mapping directly to system calls such as preadv() to minimize copies when emulating DMA. If the backing store for a device model is a raw host device (e.g. when exporting a raw disk device such as /dev/ada as a drive in the guest), the host device driver (e.g. ahci for /dev/ada) can itself use DMA on the host directly to the guest's memory. However, if the guest's memory is not present in the host IOMMU domain, these DMA requests by the host device will fail without raising an error visible to the host device driver or to the guest resulting in non-working I/O in the guest. It is unclear why guest addresses were removed from the IOMMU host domain initially, especially only for VM's with a passthru device as the host IOMMU domain does not affect the permissions of passthru devices, only devices on the host. A considered alternative was using bounce buffers instead (D34535 is a proof of concept), but that adds additional overhead for unclear benefit. This solves a long-standing problem when using passthru devices and physical disks in the same VM. Approved by:re (gjb) Thanks to: grehan (patience and help) Thanks to: jhb (for improving the commit message) PR: 260178, 215740 Reviewed by:grehan, jhb Differential Revision: https://reviews.freebsd.org/D34607 (cherry picked from commit 246c398145674e4a9337fd933a6e6da7f160118e) (cherry picked from commit dd113f67dfb5bdaf5d8b3a87bb19924ad447494c) sys/amd64/vmm/vmm.c | 2 -- 1 file changed, 2 deletions(-) -- You are receiving this mail because: You are the assignee for the bug.
[Bug 215740] [bhyve] utilizing passthru breaks raw device usage with virtio-blk | ahci-hd
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=215740 --- Comment #10 from commit-h...@freebsd.org --- A commit in branch releng/13.1 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=1c6abf864ecd3bbf07ace2018f9aab45b6406ce2 commit 1c6abf864ecd3bbf07ace2018f9aab45b6406ce2 Author: Bjoern A. Zeeb AuthorDate: 2022-03-18 20:39:06 + Commit: Bjoern A. Zeeb CommitDate: 2022-03-30 15:33:47 + bhyve: Do not remove guest physical addresses from IOMMU host domain This permits I/O devices on the host to directly access wired memory dedicated to guests using passthru devices. Note that wired memory belonging to guests that do not use passthru devices has always been accessible by I/O devices on the host. bhyve maps guest physical addresses into the user address space of the bhyve process by mmap'ing /dev/vmm/. Device models pass pointers derived from this mapping directly to system calls such as preadv() to minimize copies when emulating DMA. If the backing store for a device model is a raw host device (e.g. when exporting a raw disk device such as /dev/ada as a drive in the guest), the host device driver (e.g. ahci for /dev/ada) can itself use DMA on the host directly to the guest's memory. However, if the guest's memory is not present in the host IOMMU domain, these DMA requests by the host device will fail without raising an error visible to the host device driver or to the guest resulting in non-working I/O in the guest. It is unclear why guest addresses were removed from the IOMMU host domain initially, especially only for VM's with a passthru device as the host IOMMU domain does not affect the permissions of passthru devices, only devices on the host. A considered alternative was using bounce buffers instead (D34535 is a proof of concept), but that adds additional overhead for unclear benefit. This solves a long-standing problem when using passthru devices and physical disks in the same VM. Approved by:re (gjb) Thanks to: grehan (patience and help) Thanks to: jhb (for improving the commit message) PR: 260178, 215740 Reviewed by:grehan, jhb Differential Revision: https://reviews.freebsd.org/D34607 (cherry picked from commit 246c398145674e4a9337fd933a6e6da7f160118e) (cherry picked from commit dd113f67dfb5bdaf5d8b3a87bb19924ad447494c) sys/amd64/vmm/vmm.c | 2 -- 1 file changed, 2 deletions(-) -- You are receiving this mail because: You are the assignee for the bug.
[Bug 262920] bhyve: "/boot/userboot.so: Undefined symbol "getsecs"
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=262920 --- Comment #1 from tech-li...@zyxst.net --- The latest change involving getsecs in the timeframe is https://cgit.freebsd.org/src/commit/?h=releng/13.1&id=4003cdd81b8776cb451395ffa53423ad52328bc9 -- You are receiving this mail because: You are the assignee for the bug.
[Bug 262920] bhyve: "/boot/userboot.so: Undefined symbol "getsecs"
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=262920 tech-li...@zyxst.net changed: What|Removed |Added CC||m...@freebsd.org Flags||maintainer-feedback?(manu@f ||reebsd.org) URL||https://reviews.freebsd.org ||/D33408 -- You are receiving this mail because: You are the assignee for the bug.
[Bug 262936] bhyve randomly fails to boot/stop/reboot
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=262936 --- Comment #2 from berger...@yahoo.co.uk --- Tried this on BETA3. Haven't experienced any boot crashes yet, but neither can I restart my VM after even normal shutdown. On 13.0-RELEASE-p7 I was able to restart my VM after normal shutodwn only after bhyvectl --vm windows-10 --destroy. That was the problem on one of my machines which I learned to live with. Now this doesn't work anymore. When trying to restart after --destroy, I get this same error as mentioned earlier: bhyve: Failed to map MSI-X table BAR on 12/0/0: Device busy bhyve: failed to initialize BARs for PCI 12/0/0 device emulation initialization error: Device busy -- You are receiving this mail because: You are the assignee for the bug.
