Can you also get `xl dmesg` too, and attach it? I think this is a VirtualBox bug, but I'm confused as to why Xen has decided to turn off AVX.
~Andrew On 02/02/2025 4:01 pm, Guillaume wrote: > Yes sure I can collect the output. As you said the change is good > enough to start the dom0 without errors (at least no apparent errors :). > ``` > Xen reports there are maximum 120 leaves and 2 MSRs > Raw policy: 32 leaves, 2 MSRs > CPUID: > leaf subleaf -> eax ebx ecx edx > 00000000:ffffffff -> 00000016:756e6547:6c65746e:49656e69 > 00000001:ffffffff -> 000806c1:00020800:f6fa3203:178bfbff > 00000002:ffffffff -> 00feff01:000000f0:00000000:00000000 > 00000004:00000000 -> 04000121:02c0003f:0000003f:00000000 > 00000004:00000001 -> 04000122:01c0003f:0000003f:00000000 > 00000004:00000002 -> 04000143:04c0003f:000003ff:00000000 > 00000004:00000003 -> 04000163:02c0003f:00003fff:00000004 > 00000006:ffffffff -> 00000004:00000000:00000000:00000000 > 00000007:00000000 -> 00000000:208c2569:00000000:30000400 > 0000000b:00000000 -> 00000000:00000001:00000100:00000000 > 0000000b:00000001 -> 00000001:00000002:00000201:00000000 > 0000000d:00000000 -> 00000007:00000000:00000340:00000000 > 0000000d:00000002 -> 00000100:00000240:00000000:00000000 > 80000000:ffffffff -> 80000008:00000000:00000000:00000000 > 80000001:ffffffff -> 00000000:00000000:00000121:28100800 > 80000002:ffffffff -> 68743131:6e654720:746e4920:52286c65 > 80000003:ffffffff -> 6f432029:54286572:6920294d:31312d37 > 80000004:ffffffff -> 37473538:33204020:4730302e:00007a48 > 80000006:ffffffff -> 00000000:00000000:01007040:00000000 > 80000007:ffffffff -> 00000000:00000000:00000000:00000100 > 80000008:ffffffff -> 00003027:00000000:00000000:00000000 > MSRs: > index -> value > 000000ce -> 0000000000000000 > 0000010a -> 0000000000000000 > Host policy: 30 leaves, 2 MSRs > CPUID: > leaf subleaf -> eax ebx ecx edx > 00000000:ffffffff -> 0000000d:756e6547:6c65746e:49656e69 > 00000001:ffffffff -> 000806c1:00020800:c6fa2203:178bfbff > 00000002:ffffffff -> 00feff01:000000f0:00000000:00000000 > 00000004:00000000 -> 04000121:02c0003f:0000003f:00000000 > 00000004:00000001 -> 04000122:01c0003f:0000003f:00000000 > 00000004:00000002 -> 04000143:04c0003f:000003ff:00000000 > 00000004:00000003 -> 04000163:02c0003f:00003fff:00000004 > 00000007:00000000 -> 00000000:208c2549:00000000:30000400 > 0000000d:00000000 -> 00000003:00000000:00000240:00000000 > 80000000:ffffffff -> 80000008:00000000:00000000:00000000 > 80000001:ffffffff -> 00000000:00000000:00000121:28100800 > 80000002:ffffffff -> 68743131:6e654720:746e4920:52286c65 > 80000003:ffffffff -> 6f432029:54286572:6920294d:31312d37 > 80000004:ffffffff -> 37473538:33204020:4730302e:00007a48 > 80000006:ffffffff -> 00000000:00000000:01007040:00000000 > 80000007:ffffffff -> 00000000:00000000:00000000:00000100 > 80000008:ffffffff -> 00003027:00000000:00000000:00000000 > MSRs: > index -> value > 000000ce -> 0000000000000000 > 0000010a -> 0000000000000000 > PV Max policy: 57 leaves, 2 MSRs > CPUID: > leaf subleaf -> eax ebx ecx edx > 00000000:ffffffff -> 0000000d:756e6547:6c65746e:49656e69 > 00000001:ffffffff -> 000806c1:00020800:c6f82203:1789cbf5 > 00000002:ffffffff -> 00feff01:000000f0:00000000:00000000 > 00000004:00000000 -> 04000121:02c0003f:0000003f:00000000 > 00000004:00000001 -> 04000122:01c0003f:0000003f:00000000 > 00000004:00000002 -> 04000143:04c0003f:000003ff:00000000 > 00000004:00000003 -> 04000163:02c0003f:00003fff:00000004 > 00000007:00000000 -> 00000002:208c0109:00000000:20000400 > 0000000d:00000000 -> 00000003:00000000:00000240:00000000 > 80000000:ffffffff -> 80000021:00000000:00000000:00000000 > 80000001:ffffffff -> 00000000:00000000:00000123:28100800 > 80000002:ffffffff -> 68743131:6e654720:746e4920:52286c65 > 80000003:ffffffff -> 6f432029:54286572:6920294d:31312d37 > 80000004:ffffffff -> 37473538:33204020:4730302e:00007a48 > 80000006:ffffffff -> 00000000:00000000:01007040:00000000 > 80000007:ffffffff -> 00000000:00000000:00000000:00000100 > 80000008:ffffffff -> 00003027:00000000:00000000:00000000 > MSRs: > index -> value > 000000ce -> 0000000000000000 > 0000010a -> 0000000010020004 > HVM Max policy: 4 leaves, 2 MSRs > CPUID: > leaf subleaf -> eax ebx ecx edx > MSRs: > index -> value > 000000ce -> 0000000000000000 > 0000010a -> 0000000000000000 > PV Default policy: 30 leaves, 2 MSRs > CPUID: > leaf subleaf -> eax ebx ecx edx > 00000000:ffffffff -> 0000000d:756e6547:6c65746e:49656e69 > 00000001:ffffffff -> 000806c1:00020800:c6d82203:1789cbf5 > 00000002:ffffffff -> 00feff01:000000f0:00000000:00000000 > 00000004:00000000 -> 04000121:02c0003f:0000003f:00000000 > 00000004:00000001 -> 04000122:01c0003f:0000003f:00000000 > 00000004:00000002 -> 04000143:04c0003f:000003ff:00000000 > 00000004:00000003 -> 04000163:02c0003f:00003fff:00000004 > 00000007:00000000 -> 00000000:208c0109:00000000:20000400 > 0000000d:00000000 -> 00000003:00000000:00000240:00000000 > 80000000:ffffffff -> 80000008:00000000:00000000:00000000 > 80000001:ffffffff -> 00000000:00000000:00000121:28100800 > 80000002:ffffffff -> 68743131:6e654720:746e4920:52286c65 > 80000003:ffffffff -> 6f432029:54286572:6920294d:31312d37 > 80000004:ffffffff -> 37473538:33204020:4730302e:00007a48 > 80000006:ffffffff -> 00000000:00000000:01007040:00000000 > 80000008:ffffffff -> 00003027:00000000:00000000:00000000 > MSRs: > index -> value > 000000ce -> 0000000000000000 > 0000010a -> 0000000000000000 > HVM Default policy: 4 leaves, 2 MSRs > CPUID: > leaf subleaf -> eax ebx ecx edx > MSRs: > index -> value > 000000ce -> 0000000000000000 > 0000010a -> 0000000000000000 > ``` > > Guillaume > > On Sun, Feb 2, 2025 at 4:32 PM Andrew Cooper > <andrew.coop...@citrix.com> wrote: > > This is a sanity check that an algorithm in Xen matches hardware. > It is only compiled into debug builds by default. > > Given that you're running under virtualbox, i have a suspicion as > to what's wrong. > > Can you collect the full `xen-cpuid -p` output from within your > environment? I don't believe you're suggested code change is > correct, but it will good enough to get these diagnostics. > > ~Andrew > > On Sun, 2 Feb 2025, 15:32 Guillaume, <thouv...@gmail.com> wrote: > > Hello, > > I'd like to report an issue I encountered when building Xen > from source. To give you some context, During the Xen winter > meetup in Grenoble few days ago, there was a discussion about > strengthening collaboration between Xen and academia. One > issue raised by a professor was that Xen is harder for > students to install and experiment compared to KVM. In > response it was mentionned that Debian packages are quite > decent. This motivated me to try installing and playing with > Xen myself. While I am familiar with Xen (I work on the XAPI > toolstack at Vates) I'm not deeply familiar with its > internals, so this seemed like a good learning opportunity and > maybe some contents for some blog posts :). > > I set up a Debian testing VM on Virtualbox and installed Xen > from packages. Everything worked fine: Grub was updated, I > rebooted, and I had a functional Xen setup with xl running in > Dom0. > Next I download the last version of Xen from xenbits.org > <http://xenbits.org>,and built only the hypervisor (no tools, > no stubdom) , using the same configuration as the Debian > package (which is for Xen 4.19). After updating GRUB and > rebooting, Xen failed to boot. Fortunately, I was able to > capture the following error via `ttyS0`: > ``` > (XEN) [0000000d2c23739a] xstate: size: 0x340 and states: 0x7 > (XEN) [0000000d2c509c1d] > (XEN) [0000000d2c641ffa] **************************************** > (XEN) [0000000d2c948e3b] Panic on CPU 0: > (XEN) [0000000d2cb349bb] XSTATE 0x0000000000000003, > uncompressed hw size 0x340 != xen size 0x240 > (XEN) [0000000d2cfc5786] **************************************** > (XEN) [0000000d2d308c24] > ``` > From my understanding, the hardware xstate size (`hw_size`) > represents the maximum memory required for the `XSAVE/XRSTOR` > save area, while `xen_size` is computed by summing the space > required for the enabled features. In `xen/arch/x86/xstate.c`, > if these sizes do not match, Xen panics. However, wouldn’t it > be correct for `xen_size` to be **less than or equal to** > `hw_size` instead of exactly matching? > > I tested the following change: > ``` > --- a/xen/arch/x86/xstate.c > +++ b/xen/arch/x86/xstate.c > @@ -710,7 +710,7 @@ static void __init check_new_xstate(struct > xcheck_state *s, uint64_t new) > */ > xen_size = xstate_uncompressed_size(s->states & > X86_XCR0_STATES); > > - if ( xen_size != hw_size ) > + if ( xen_size > hw_size ) > panic("XSTATE 0x%016"PRIx64", uncompressed hw size > %#x != xen size %#x\n", > s->states, hw_size, xen_size); > ``` > With this change, Xen boots correctly, but I may be missing > some side effects... > Additionally, I am confused as to why this issue does *not* > occur with the default Debian Xen package. Even when I rebuild > Xen *4.19.1* from source (the same version as the package), I > still encounter the issue. > So I have two questions: > - Is my understanding correct that |xen_size <= hw_size| > should be allowed? > - Are there any potential side effects of this change? > - Bonus: Have some of you any explanations about why does the > issue not occur with the packaged version of Xen but does with > a self-built version? > > Hope I wasn't too long and thanks for taking the time to read > this, > Best regards, > > Guillaume >
