Hi Vincent, thanks for your report - really good research, even indicating a
potential fix.
I manage to reproduce this issue using the following steps:
1) Be sure to run Ubuntu 16.04 on both guest and host, both
running kernel 4.4.0 (could be latest version of this kernel)
2) In the guest, install the package "msr-tools", like:
"sudo apt install msr-tools"
3) Still on guest, run: "sudo modprobe msr; sudo wrmsr 0x48 1"
This will enable the MSR bit for IBRS inside the guest
4) Now the host shows bad performance
If we run "sudo wrmsr 0x48 0" in the guest, host gets its performance
back.
I'll investigate some commits upstream, including the one you suggested, and
once we figure the exact fix for this, will request SRU to the kernel team.
Thanks,
Guilherme
** Changed in: linux (Ubuntu)
Assignee: (unassigned) => Guilherme G. Piccoli (gpiccoli)
** Changed in: linux (Ubuntu Xenial)
Assignee: (unassigned) => Guilherme G. Piccoli (gpiccoli)
** Tags added: sts
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1764956
Title:
Guests using IBRS incur a large performance penalty
Status in linux package in Ubuntu:
Triaged
Status in linux source package in Xenial:
Triaged
Bug description:
Hello!
As of Linux 4.4.0-119, when a KVM guest is using IBRS, this incurs a
very large performance penalty on the hosts and other guests.
From my understanding, the patch
f676aa34b4027d1a7a4bbcc58b81b20c68c7ce0c is incomplete. If host
doesn't handle IBRS itself (which is now the case by default since
4.4.0-116: it relies on retpoline instead) but the guest does (eg
running an earlier kernel), the guest will set IBRS for the CPU it is
running on from time to time but if it gets preempted at some point,
the IBRS bit will stay, incurring a major performance penalty for all
other users of the CPU (host userland, host kernel and other guests
not caring about IBRS). The equivalent patch in mainline
(d28b387fb74da95d69d2615732f50cceb38e9a4d) ensure the appropriate MSR
is correctly restored when switching from one guest to another or from
one guest to host.
The issue is easy to reproduce: host running 4.4.0-119, exposing
"spec_ctrl" to a guest running CentOS 7.4 with its January kernel.
Wait a few minutes and the host will become pretty slow. A simple
shell loop will take 10 more times to execute. Executing "sysctl -w
kernel.ibrs_dump=1" will show that most real cores have now their IBRS
bit set to 1.
A workaround is to reeanble IBRS on the host (sysctl -w
kernel.ibrs_enabled=1). This way, IBRS will be correctly disabled when
changing context.
A long term solution would be to properly backport the patch from
mainline. It is not part of the 4.4 stable branch and it seems not
trivial to port.
A mid term solution could be to remove the faulty patch (not exposing
IBRS), since most VM don't need it anymore. This also salvage the
ability to use IBPB (which doesn't seem to alter performance that
much) but it isn't believed to be essential.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1764956/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
