Re: native_cpu_up speed (Re: [PATCH] x86: skip delays during SMP initialization similar to Xen)

On 05/29/2015 09:47 AM, Len Brown wrote: However, a clear pattern jumped out of the trace for how long the BSP waits for the AP to set itself in cpu_callin_mask. This is the time in start secondary where cpu_init() is running, up through smp_callin() is called. On the 1st package, each remote AP

native_cpu_up speed (Re: [PATCH] x86: skip delays during SMP initialization similar to Xen)

>> I don't know if anything can be done for the 1700us wait >> for the remote processor to mark itself initialized. >> That is the 1st thing it does when it enters cpu_init(). > > So that 1.7 msecs delay is the firmware in essence? Yes -- hardware+microcode+firmware initialization. I measured thi

check_tsc_time_warp() (was Re: [PATCH] x86: skip delays during SMP initialization similar to Xen)

On Sun, May 17, 2015 at 1:26 AM, Ingo Molnar wrote: >> check_tsc_warp() is hard-coded to take 2ms. I don't know if 2ms is a >> magic number or if shorter has same value. It seems a bit sad to do >> this serially for every CPU at boot, when we could do all the CPUs >> in parallel after they are on

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

>> this time can be reduced by 7% (113 ms) by deleting announce_cpu() While the KERN_DEBUG output is fast, accounce_cpu() uses KERN_INFO, which goes to the (serial) console by default. One would expect it to drain at about 10 bits/byte / 115200 baud = 87us/byte. I measured some vanilla printk's v

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

>> >> BTW. this time can be reduced by 7% (113 ms) by deleting >> >> announce_cpu(): >> >> >> >> [1.445815] x86: Booted up 4 nodes, 120 CPUs >> > >> > so that kind of info looks pretty useful, especially when there's >> > hangs/failures. >> >> I think the messages we print on failure are useful

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

* Len Brown wrote: > On Thu, May 14, 2015 at 1:57 PM, Ingo Molnar wrote: > > > > * "Jan H. Schönherr" wrote: > > > >> Ingo, do you want an updated version of the original patch, which > >> takes care not get stuck, when the INIT deassertion is skipped, or > >> do you prefer to address delays "

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

* Len Brown wrote: > On Thu, May 14, 2015 at 2:44 AM, Ingo Molnar wrote: > > > > >> BTW. this time can be reduced by 7% (113 ms) by deleting > >> announce_cpu(): > >> > >> [1.445815] x86: Booted up 4 nodes, 120 CPUs > > > > so that kind of info looks pretty useful, especially when there's

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

On Thu, May 14, 2015 at 1:57 PM, Ingo Molnar wrote: > > * "Jan H. Schönherr" wrote: > >> Ingo, do you want an updated version of the original patch, which >> takes care not get stuck, when the INIT deassertion is skipped, or >> do you prefer to address delays "one by one" as you wrote elsewhere?

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

On Thu, May 14, 2015 at 2:44 AM, Ingo Molnar wrote: > >> BTW. this time can be reduced by 7% (113 ms) by deleting >> announce_cpu(): >> >> [1.445815] x86: Booted up 4 nodes, 120 CPUs > > so that kind of info looks pretty useful, especially when there's > hangs/failures. I think the messages

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

* "Jan H. Schönherr" wrote: > Ingo, do you want an updated version of the original patch, which > takes care not get stuck, when the INIT deassertion is skipped, or > do you prefer to address delays "one by one" as you wrote elsewhere? So I'm not against improving this code at all, but instea

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

On 05/14/2015 09:18 AM, Len Brown wrote: On Thu, May 14, 2015 at 2:36 AM, Len Brown wrote: [2.737884] x86: Booted up 4 nodes, 120 CPUs For the record, the same (bare metal) box running latest tip boots 10ms/processor quicker [1.553658] x86: Booted up 4 nodes, 120 CPUs BTW. thi

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

On Thu, May 14, 2015 at 2:36 AM, Len Brown wrote: >> [2.737884] x86: Booted up 4 nodes, 120 CPUs > For the record, the same (bare metal) box running latest tip boots > 10ms/processor quicker > [1.553658] x86: Booted up 4 nodes, 120 CPUs > BTW. this time can be reduced by 7% (113 ms) by

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

* Len Brown wrote: > > [0.404369] x86: Booting SMP configuration: > ... > > [2.737884] x86: Booted up 4 nodes, 120 CPUs > > [2.743758] smpboot: Total of 120 processors activated (671097.18 > > BogoMIPS) > > > > (2.743758-0.404369) = 2.339389 for all 119 processors > > /119 = .019658

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

> [0.404369] x86: Booting SMP configuration: ... > [2.737884] x86: Booted up 4 nodes, 120 CPUs > [2.743758] smpboot: Total of 120 processors activated (671097.18 BogoMIPS) > > (2.743758-0.404369) = 2.339389 for all 119 processors > /119 = .01965873109243697478 - lets call it 19ms each

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

On Tue, May 12, 2015 at 11:47:09AM +0200, Ingo Molnar wrote: > So I was booting 120 CPUs with kvmtool (tools/kvm/ under -tip). > > Even with your patches applied it's 7 seconds (config attached): > > [0.152189] x86: Booting SMP configuration: > [0.152705] node #0, CPUs: #1

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

* Len Brown wrote: > > It was 20+ seconds before that, 10+ seconds for the SMP bootup > > sequence. > > (1.625928-0.558947) = 1.07 seconds to online 119 additional cpus. > /119 = .0089662268 each, lets call it 9ms. > > Here is my ivb-ex running stock fedora 21's Linux-3.19 (no patch applied):

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

* Len Brown wrote: > On Thu, May 7, 2015 at 6:23 AM, Ingo Molnar wrote: > > These numbers do not look right. > > > Here's what the boot time looks like on a 120 CPUs system, with the > > patch applied: > > > > [0.558947] x86: Booting SMP configuration: > > [0.563375] node #0, CP

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

On Thu, May 7, 2015 at 6:23 AM, Ingo Molnar wrote: These numbers do not look right. > Here's what the boot time looks like on a 120 CPUs system, with the > patch applied: > > [0.558947] x86: Booting SMP configuration: > [0.563375] node #0, CPUs: #1 #2 #3 #4 #5 #6

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

* Len Brown wrote: > > So I really like this, as it nicely side-steps the 'when should we > > do the legacy delays' issue by flagging on x2apic support. > > > > If anyone has objections, please holler. > > We should have no delays even for many processors that lack x2apic. Yes, agreed, but th

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

> So I really like this, as it nicely side-steps the 'when should we do > the legacy delays' issue by flagging on x2apic support. > > If anyone has objections, please holler. We should have no delays even for many processors that lack x2apic. Len Brown, Intel Open Source Technology Center -- To u

Re: [PATCH] x86: skip delays during SMP initialization similar to Xen

* Jan H. Schönherr wrote: > Remove the per-CPU delays during SMP initialization, which seems to be > possible on newer architectures with an x2APIC. > > Xen does this since 2011. In fact, this commit is basically a > combination of the following Xen commits. The first removes the delays, > the

[PATCH] x86: skip delays during SMP initialization similar to Xen

Remove the per-CPU delays during SMP initialization, which seems to be possible on newer architectures with an x2APIC. Xen does this since 2011. In fact, this commit is basically a combination of the following Xen commits. The first removes the delays, the second fixes an issue with the removal: