Public bug reported:
[Impact]
When kexec'ing into a crash kernel with `ncpus` > 1, VMs can raise a KVM
emulation failure. This will cause the VM to go into the "paused" state, and
prevents it from being restored without a full VM restart.
This happens only when there are multiple enabled CPUs in the crash kernel
command-line, regardless of whether `nr_cpus` or `maxcpus` is being used. Due to
the vCPU MMU state not being cleaned up correctly, the secondary CPUs try to
access virtual addresses with a faulty MMU context that will result in the
emulation failure. This shows up with a similar spew as below:
$ sudo tail -n20 /var/log/libvirt/qemu/focal-vm.log
KVM internal error. Suberror: 1
emulation failure
EAX=0000de8f EBX=00000000 ECX=0000008f EDX=00000600
ESI=00000000 EDI=00000000 EBP=00000000 ESP=0000f90c
EIP=0000cdb1 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 0000ffff 00009300
CS =f000 000f0000 0000ffff 00009b00
SS =de00 000de000 0000ffff 00009300
DS =de00 000de000 0000ffff 00009300
FS =0000 00000000 0000ffff 00009300
GS =0000 00000000 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT= 00000000 0000ffff
IDT= 00000000 0000ffff
CR0=60000010 CR2=00000000 CR3=290b8001 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=66 83 c4 28 66 5b 66 c3 66 56 66 53 66 52 b1 8f 88 c8 e6 70 <e4> 71 66 0f
b6 f0 66 89 f2 67 88 54 24 03 88 c8 e6 70 66 31 db 88 d8 e6 71 66 56 66 68 1a
[Test Plan]
1. Boot an Ubuntu guest VM with e.g. multipass:
$ multipass launch daily:focal -c8 -m16g -n focal-vm
2. Configure guest crash kernel command-line with `nr_cpus=8`:
ubuntu@focal-vm:~$ grep CMDLINE_APPEND /etc/default/kdump-tools
# KDUMP_CMDLINE_APPEND - Additional arguments to append to the command line
KDUMP_CMDLINE_APPEND="reset_devices systemd.unit=kdump-tools-dump.service
nr_cpus=8 irqpoll nousb ata_piix.prefer_ms_hyperv=0"
3. Crash guest VM and watch for the KVM emulation failure:
ubuntu@focal-vm:~$ echo c | sudo tee /proc/sysrq-trigger
[Where problems could occur]
As we're resetting MMU context on vCPUs, potential regressions would show up in
workloads relying on KVM guests. We should properly test the scenario mentioned
in the bug to make sure secondary CPUs are being cleaned up properly, and that
no other regressions have been introduced when rebooting or kexec'ing into
different kernels.
Since we're adding an MMU reset at kvm_vcpu_reset(), the overall regression
potential should be fairly low and contained to starting/resetting vCPUs
(i.e. VM start and reboot).
[Other info]
This has been fixed by upstream commit:
0aa1837533e5 KVM: x86: Properly reset MMU context at vCPU RESET/INIT
And the two follow up commits, which revert the vendor-specific resets:
5d2d7e41e3b8 KVM: SVM: Drop explicit MMU reset at RESET/INIT
61152cd907d5 KVM: VMX: Remove explicit MMU reset in enter_rmode()
These commits have been introduced during the upstream 5.14 and 5.15 release
candidates, and as such should be backported to previous supported kernels.
$ git describe --contains 0aa1837533e5 5d2d7e41e3b8 61152cd907d5
v5.14-rc1~166^2~58
v5.15-rc1~65^2~119
v5.15-rc1~65^2~120
** Affects: linux (Ubuntu)
Importance: High
Assignee: Heitor Alves de Siqueira (halves)
Status: New
** Tags: sts
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1948862
Title:
KVM emulation failure when booting into VM crash kernel with multiple
CPUs
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1948862/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
