Hello David, thank you for the feedback. On Wed, Feb 2, 2022 at 12:47 PM David Edmondson <david.edmond...@oracle.com> wrote: > > On Tuesday, 2022-02-01 at 16:09:57 -03, Leonardo Brás wrote: > > > Hello David, thanks for this feedback! > > > > On Mon, 2022-01-31 at 12:53 +0000, David Edmondson wrote: > >> On Saturday, 2022-01-29 at 06:46:45 -03, Leonardo Bras wrote: > >> > >> > The following steps describe a migration bug: > >> > 1 - Bring up a VM with -cpu EPYC on a host with EPYC-Milan cpu > >> > 2 - Migrate to a host with EPYC-Naples cpu > >> > > >> > The guest kernel crashes shortly after the migration. > >> > > >> > The crash happens due to a fault caused by XRSTOR: > >> > A set bit in XSTATE_BV is not set in XCR0. > >> > The faulting bit is FEATURE_PKRU (enabled in Milan, but not in > >> > Naples) > >> > >> I'm trying to understand how this happens. > >> > >> If we boot on EPYC-Milan with "-cpu EPYC", the PKRU feature should > >> not > >> be exposed to the VM (it is not available in the EPYC CPU). > >> > >> Given this, how would bit 0x200 (representing PKRU) end up set in > >> xstate_bv? > > > > During my debug, I noticed this bit gets set before the kernel even > > starts. > > > > It's possible Seabios and/or IPXE are somehow setting 0x200 using the > > xrstor command. I am not sure if qemu is able to stop this in KVM mode. > > I don't believe that this should be possible. > > If the CPU is set to EPYC in QEMU then .features[FEAT_7_0_ECX] does not > include CPUID_7_0_ECX_PKU, which in turn means that when > x86_cpu_enable_xsave_components() generates FEAT_XSAVE_COMP_LO it should > not set XSTATE_PKRU_BIT. > > Given that, KVM's vcpu->arch.guest_supported_xcr0 will not include > XSTATE_PKRU_BIT, and __kvm_set_xcr() should not allow that bit to be > set when it intercepts the guest xsetbv instruction.
Thanks for sharing those details, it helped me on the kernel side of this bug. FWIW, i did send a patchset fixing this bug to kernel list: https://patchwork.kernel.org/project/kvm/list/?series=611524&state=%2A&archive=both Best regards, Leo
