On 30/09/2019 23:01, Roman Shaposhnik wrote: > On Mon, Sep 30, 2019 at 3:19 AM Andrew Cooper <andrew.coop...@citrix.com> > wrote: >> On 28/09/2019 04:07, Roman Shaposhnik wrote: >>> On Thu, Sep 26, 2019 at 12:44 AM Jan Beulich <jbeul...@suse.com> wrote: >>>> On 26.09.2019 00:31, Roman Shaposhnik wrote: >>>>> Jan, Roger, thank you so much for the initial ideas. I tried a few of >>>>> those and here's where I am. >>>>> >>>>> First of all, it is definitely related to CPU bring up. Adding >>>>> cpuidle=0 to xen command line made Xen boot. >>>>> >>>>> Then, a good friend of mine (who you may know from ancient Xen days >>>>> ;-)) suggested that this could be related to this: >>>>> https://wiki.xenproject.org/wiki/Xen_power_management >>>>> so I went to the BIOS settings and quite to my surprise all of them >>>>> were grayed out (not tweakable). >>>>> >>>>> The only one that wasn't was 2xAPIC support. So just for kicks -- I >>>>> disabled that. >>>>> >>>>> That, in turn, made Xen boot even without cpuidle=0. I'm attaching that >>>>> log. >>>> Interesting, but unfortunately this particular log is of no real use >>>> for investigation of the issue (other than knowing the CPU model). I >>>> also notice it's a 4.12.0 log, when your original report was against >>>> latest master. >>>> >>>>> So I guess at this point, you could say that I have a functional >>>>> system, but I'm curious whether you guys would be interested to look >>>>> into 2xAPIC situation. >>>> Of course we do. As a next step I'd suggest reverting the BIOS settings >>>> change you did, and instead using the "x2apic=0" Xen command line option. >>> Interestingly enough, this doesn't really solve the problem completely. >>> Specifying x2apic=0 certainly makes Xen go much further to a point >>> where it tries to load Dom0 and then the console VGA screen goes >>> blank (this is where that serial debug output would be very useful :-(). >> Do you not even have IPMI SoL? Its basically impossible to buy servers >> without this base level of functionality. >> >> It will be somewhere in the iLO configuration on an HP server. > This is a super awesome suggestion! > > Attached are two logs. Please take a look -- both result in Xen hanging.
That's good to hear. Hopefully we can make some progress. To begin with, try dropping "dom0_vcpus_pin smt=false" from the command line. Both may hit some corner cases. Because of "(XEN) x2APIC mode is already enabled by BIOS.", the x2apic=0 part is useless and will be ignored. It can be dropped too. Can you rebase to using Xen 4.12.1? It has a substantial number of bugfixes over 4.12.0, and should strictly be used in preference. Alternatively, staging-4.12 has a few fixes more than 4.12.1, notably "x86/apic: enable x2APIC mode before doing any setup" which fixes an issue with x2APIC initialisation. If possible, can you use a debug build of Xen? ~Andrew _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
