Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-20 Thread Mel Gorman
On Fri, Dec 20, 2013 at 11:51:43PM +0800, Fengguang Wu wrote: > On Thu, Dec 19, 2013 at 02:34:50PM +, Mel Gorman wrote: > > On Wed, Dec 18, 2013 at 03:28:14PM +0800, Fengguang Wu wrote: > > > Hi Mel, > > > > > > I'd like to share some test numbers with your patches applied on top of > > > v3.

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-20 Thread Mel Gorman
On Fri, Dec 20, 2013 at 01:20:19PM +0100, Ingo Molnar wrote: > > * Mel Gorman wrote: > > > tlb_flushall_shift == -1Always use flush all > > tlb_flushall_shift == 1 Aggressively use individual flushes > > tlb_flushall_shift == 6 Conservatively use individual flushes >

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-20 Thread Ingo Molnar
* Mel Gorman wrote: > tlb_flushall_shift == -1 Always use flush all > tlb_flushall_shift == 1 Aggressively use individual flushes > tlb_flushall_shift == 6 Conservatively use individual flushes > > IvyBridge was too aggressive using individual flushes and my pat

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-20 Thread Mel Gorman
On Fri, Dec 20, 2013 at 12:18:18PM +0100, Ingo Molnar wrote: > > * Mel Gorman wrote: > > > On Thu, Dec 19, 2013 at 05:49:25PM +0100, Ingo Molnar wrote: > > > > > > * Mel Gorman wrote: > > > > > > > [...] > > > > > > > > Because we lack data on TLB range flush distributions I think we > > >

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-20 Thread Ingo Molnar
* Mel Gorman wrote: > On Thu, Dec 19, 2013 at 05:49:25PM +0100, Ingo Molnar wrote: > > > > * Mel Gorman wrote: > > > > > [...] > > > > > > Because we lack data on TLB range flush distributions I think we > > > should still go with the conservative choice for the TLB flush > > > shift. The

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-20 Thread Mel Gorman
On Thu, Dec 19, 2013 at 05:49:25PM +0100, Ingo Molnar wrote: > > * Mel Gorman wrote: > > > [...] > > > > Because we lack data on TLB range flush distributions I think we > > should still go with the conservative choice for the TLB flush > > shift. The worst case is really bad here and it's pa

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-19 Thread Ingo Molnar
* Mel Gorman wrote: > [...] > > Because we lack data on TLB range flush distributions I think we > should still go with the conservative choice for the TLB flush > shift. The worst case is really bad here and it's painfully obvious > on ebizzy. So I'm obviously much in favor of this - I'd i

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-19 Thread Mel Gorman
On Wed, Dec 18, 2013 at 03:28:14PM +0800, Fengguang Wu wrote: > Hi Mel, > > I'd like to share some test numbers with your patches applied on top of > v3.13-rc3. > > Basically there are > > 1) no big performance changes > > 76628486 -0.7% 76107841 TOTAL vm-scalability.throug

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-19 Thread Mel Gorman
On Tue, Dec 17, 2013 at 12:00:51PM +0100, Ingo Molnar wrote: > > It's eliminated for one machine and reduced for another. > > > > Signed-off-by: Mel Gorman > > --- > > kernel/sched/core.c | 5 +++-- > > 1 file changed, 3 insertions(+), 2 deletions(-) > > > > diff --git a/kernel/sched/core.c b/k

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-18 Thread Ingo Molnar
* Mel Gorman wrote: > > Thanks again for going through all this. Tracking multi-commit > > performance regressions across 1.5 years worth of commits is > > generally very hard. Does your testing effort comes from > > enterprise Linux QA testing, or did you ran into this problem > > accidenta

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-17 Thread Fengguang Wu
Hi Mel, I'd like to share some test numbers with your patches applied on top of v3.13-rc3. Basically there are 1) no big performance changes 76628486 -0.7% 76107841 TOTAL vm-scalability.throughput 407038 +1.2% 412032 TOTAL hackbench.throughput 5

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-17 Thread Mel Gorman
On Tue, Dec 17, 2013 at 03:42:14PM +0100, Ingo Molnar wrote: > > * Mel Gorman wrote: > > > [...] > > > > At that point it'll be time to look at profiles and see where we are > > actually spending time because the possibilities of finding things > > to fix through bisection will be exhausted. >

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-17 Thread Ingo Molnar
* Mel Gorman wrote: > [...] > > At that point it'll be time to look at profiles and see where we are > actually spending time because the possibilities of finding things > to fix through bisection will be exhausted. Yeah. One (heavy handed but effective) trick that can be used in such a sit

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-17 Thread Mel Gorman
On Tue, Dec 17, 2013 at 12:00:51PM +0100, Ingo Molnar wrote: > > > sched: Assign correct scheduling domain to sd_llc > > > > Commit 42eb088e (sched: Avoid NULL dereference on sd_busy) corrected a NULL > > dereference on sd_busy but the fix also altered what scheduling domain it > > used for sd_ll

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-17 Thread Ingo Molnar
* Mel Gorman wrote: > On Mon, Dec 16, 2013 at 02:44:49PM +0100, Ingo Molnar wrote: > > > > * Mel Gorman wrote: > > > > > > Whatever we did right in v3.4 we want to do in v3.13 as well - or > > > > at least understand it. > > > > > > Also agreed. I started a bisection before answering this m

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-17 Thread Mel Gorman
On Mon, Dec 16, 2013 at 09:17:35AM -0800, Linus Torvalds wrote: > On Mon, Dec 16, 2013 at 2:39 AM, Mel Gorman wrote: > > > > First was Alex's microbenchmark from https://lkml.org/lkml/2012/5/17/59 > > and ran it for a range of thread numbers, 320 iterations per thread with > > random number of ent

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-17 Thread Peter Zijlstra
On Tue, Dec 17, 2013 at 09:21:25AM +, Mel Gorman wrote: > if (sd) { > id = cpumask_first(sched_domain_span(sd)); > size = cpumask_weight(sched_domain_span(sd)); > - sd = sd->parent; /* sd_busy */ > + busy_sd = sd->parent; /* sd_busy */ >

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-17 Thread Mel Gorman
On Mon, Dec 16, 2013 at 02:44:49PM +0100, Ingo Molnar wrote: > > * Mel Gorman wrote: > > > > Whatever we did right in v3.4 we want to do in v3.13 as well - or > > > at least understand it. > > > > Also agreed. I started a bisection before answering this mail. It > > would be cooler and potent

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-16 Thread Linus Torvalds
On Mon, Dec 16, 2013 at 2:39 AM, Mel Gorman wrote: > > First was Alex's microbenchmark from https://lkml.org/lkml/2012/5/17/59 > and ran it for a range of thread numbers, 320 iterations per thread with > random number of entires to flush. Results are from two machines There's something wrong with

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-16 Thread Ingo Molnar
* Mel Gorman wrote: > > Whatever we did right in v3.4 we want to do in v3.13 as well - or > > at least understand it. > > Also agreed. I started a bisection before answering this mail. It > would be cooler and potentially faster to figure it out from direct > analysis but bisection is reliab

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-16 Thread Mel Gorman
On Mon, Dec 16, 2013 at 11:24:39AM +0100, Ingo Molnar wrote: > > * Mel Gorman wrote: > > > I had hacked ebizzy to report on the performance of each thread, not > > just the overall result and worked out the difference in performance > > of each thread. In a complete fair test you would expect

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-16 Thread Mel Gorman
On Sun, Dec 15, 2013 at 10:34:25AM -0800, Linus Torvalds wrote: > On Sun, Dec 15, 2013 at 7:55 AM, Mel Gorman wrote: > > > > Short answer -- There appears to be a second bug where 3.13-rc3 is less > > fair to threads getting time on the CPU. > > Hmm. Can you point me at the (fixed) microbenchmar

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-16 Thread Mel Gorman
On Fri, Dec 13, 2013 at 02:38:32PM -0800, H. Peter Anvin wrote: > On 12/13/2013 01:16 PM, Linus Torvalds wrote: > > On Fri, Dec 13, 2013 at 12:01 PM, Mel Gorman wrote: > >> > >> ebizzy > >> 3.13.0-rc33.4.69 > >> 3.13.0-rc33.13.0-rc3 > >

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-16 Thread Ingo Molnar
* Mel Gorman wrote: > I had hacked ebizzy to report on the performance of each thread, not > just the overall result and worked out the difference in performance > of each thread. In a complete fair test you would expect the > performance of each thread to be identical and so the spread would

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-15 Thread Linus Torvalds
On Sun, Dec 15, 2013 at 7:55 AM, Mel Gorman wrote: > > Short answer -- There appears to be a second bug where 3.13-rc3 is less > fair to threads getting time on the CPU. Hmm. Can you point me at the (fixed) microbenchmark you mention? Linus -- To unsubscribe from this list: send t

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-15 Thread Mel Gorman
On Sun, Dec 15, 2013 at 03:55:39PM +, Mel Gorman wrote: > > tlbflush micro benchmark > 3.13.0-rc33.13.0-rc33.4.69 >vanilla nowalk-v2r7 vanilla > Min17.00 ( 0.00%)6.00 ( 14.29%)

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-15 Thread Mel Gorman
On Fri, Dec 13, 2013 at 01:16:41PM -0800, Linus Torvalds wrote: > On Fri, Dec 13, 2013 at 12:01 PM, Mel Gorman wrote: > > > > ebizzy > > 3.13.0-rc33.4.69 > > 3.13.0-rc33.13.0-rc3 > > thread vanilla vanill

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-13 Thread H. Peter Anvin
On 12/13/2013 01:16 PM, Linus Torvalds wrote: > On Fri, Dec 13, 2013 at 12:01 PM, Mel Gorman wrote: >> >> ebizzy >> 3.13.0-rc33.4.693.13.0-rc3 >>3.13.0-rc3 >> thread vanilla vanilla altershift-v2r1

Re: [PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-13 Thread Linus Torvalds
On Fri, Dec 13, 2013 at 12:01 PM, Mel Gorman wrote: > > ebizzy > 3.13.0-rc33.4.693.13.0-rc3 > 3.13.0-rc3 > thread vanilla vanilla altershift-v2r1 > nowalk-v2r7 > Mean 1 7377.91 (

[PATCH 0/4] Fix ebizzy performance regression due to X86 TLB range flush v2

2013-12-13 Thread Mel Gorman
Changelog since v1 o Drop a pagetable walk that seems redundant o Account for TLB flushes only when debugging o Drop the patch that took number of CPUs to flush into account ebizzy regressed between 3.4 and 3.10 while testing on a new machine. Bisection initially found at least three problems of w