Hello, We have some issues on using bhyve from inside a FreeBSD KVM virtual machine on Intel E5-2660 v4 servers. The host operating systems we tried and the errors are described below.
If the host is one of the following operating systems: Ubuntu 18.04, Ubuntu 18.04 for servers or CentOS8 we get the following behavior: Tested on FreeBSD-Current (FreeBSD-13.0-CURRENT-amd64-20200213-r357847) and FreeBSD12.1: - starting bhyve using bhyveload as a loader ends with a kernel panic in the bhyve VM due to APIC error: # sh /usr/share/examples/bhyve/vmrun.sh -c 1 -m 1G -d guest.img -i -I FreeBSD-12.1-RELEASE-amd64-disc1.iso machine ==== Event timer "LAPIC" quality 600 ACPI APIC Table: <BHYVE BVMADT > panic: APIC: CPU with APIC ID 255 is not enabled cpuid = 0 time = 1 KDB: stack backtrace: #0 0xffffffff80c1d297 at kdb_backtrace+0x67 #1 0xffffffff80bd05cd at vpanic+0x19d #2 0xffffffff80bd0423 at panic+0x43 #3 0xffffffff811f7569 at madt_set_ids+0x99 #4 0xffffffff80b66918 at mi_startup+0x118 #5 0xffffffff8036b02c at btext+0x2c Uptime: 1s ==== - starting bhyve using uefi as a loader seems to stuck or run in an infinite loop - nothing is displayed on screen after starting the VM, we cannot interact with it. bhyve -c 1 -m 1G -H -P -S -s 0:0,hostbridge -s 1:0,lpc -s 4:0,ahci-hd,guest.img -s 3:0,ahci-cd,<ISO> -l com1,stdio -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd <vmname> *stuck - does not receive input, does not display anything. We also tried disabling x2apic from the host kernel options and also from the KVM guest options. As for libvirt options, we tried both cpu options: host-model and host-passthrough. If the host is a CentOS7, we didn't manage to get the FreeBSD KVM virtual machine to see the hardware virtualization features and we cannot run `kldload vmm` successfully (error: vmx_init: processor does not support desired basic capabilities). We have tested with a CentOS7 guest and nested virtualization with kvm does not have any issues. In the document at [1], we centralized the tests we run and their output, including virsh xml files, bhyve commands, and the errors we have encounters. As a parallel, we installed FreeBSD directly on the server and ran the same bhyve commands that worked fine. Does anyone know any way to solve this issue? [1] https://docs.google.com/document/d/1DIzdJKHNZNLZXk9dJquUDwwYJpTH2zDkT9eF2sVIb-A/edit?usp=sharing Thank you, Elena _______________________________________________ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
