Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-19 Thread Jason Low
On Fri, 2013-07-19 at 20:37 +0200, Peter Zijlstra wrote: > On Thu, Jul 18, 2013 at 12:06:39PM -0700, Jason Low wrote: > > > N = 1 > > - > > 19.21% reaim [k] __read_lock_failed > > 14.79% reaim [k] mspin_lock > > 12.19% reaim [k] __write_l

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-19 Thread Peter Zijlstra
On Thu, Jul 18, 2013 at 12:06:39PM -0700, Jason Low wrote: > N = 1 > - > 19.21% reaim [k] __read_lock_failed > 14.79% reaim [k] mspin_lock > 12.19% reaim [k] __write_lock_failed > 7.87% reaim [k] _raw_spin_lock

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-18 Thread Jason Low
On Thu, 2013-07-18 at 07:59 -0400, Rik van Riel wrote: > On 07/18/2013 05:32 AM, Peter Zijlstra wrote: > > On Wed, Jul 17, 2013 at 09:02:24PM -0700, Jason Low wrote: > > > >> I ran a few AIM7 workloads for the 8 socket HT enabled case and I needed > >> to set N to more than 20 in order to get the b

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-18 Thread Jason Low
On Thu, 2013-07-18 at 17:42 +0530, Srikar Dronamraju wrote: > > > > > > idle_balance(u64 idle_duration) > > > { > > > u64 cost = 0; > > > > > > for_each_domain(sd) { > > > if (cost + sd->cost > idle_duration/N) > > > break; > > > > > > ... > > > > > > sd->cost = (sd->cost

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-18 Thread Srikar Dronamraju
* Peter Zijlstra [2013-07-18 14:35:31]: > On Thu, Jul 18, 2013 at 05:45:46PM +0530, Srikar Dronamraju wrote: > > We take locks if and only if we see imbalance and want to pull the > > tasks. > > However if the newly idle balance is not finding an imbalance then this > > may not be an issue. > >

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-18 Thread Peter Zijlstra
On Thu, Jul 18, 2013 at 05:45:46PM +0530, Srikar Dronamraju wrote: > We take locks if and only if we see imbalance and want to pull the > tasks. > However if the newly idle balance is not finding an imbalance then this > may not be an issue. > > Probably /proc/schedstats will give a better picture

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-18 Thread Srikar Dronamraju
* Rik van Riel [2013-07-18 07:59:22]: > On 07/18/2013 05:32 AM, Peter Zijlstra wrote: > >On Wed, Jul 17, 2013 at 09:02:24PM -0700, Jason Low wrote: > > > >>I ran a few AIM7 workloads for the 8 socket HT enabled case and I needed > >>to set N to more than 20 in order to get the big performance gai

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-18 Thread Srikar Dronamraju
> > > > idle_balance(u64 idle_duration) > > { > > u64 cost = 0; > > > > for_each_domain(sd) { > > if (cost + sd->cost > idle_duration/N) > > break; > > > > ... > > > > sd->cost = (sd->cost + this_cost) / 2; > > cost += this_cost; > > } > > } > > > > I would've initi

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-18 Thread Rik van Riel
On 07/18/2013 05:32 AM, Peter Zijlstra wrote: On Wed, Jul 17, 2013 at 09:02:24PM -0700, Jason Low wrote: I ran a few AIM7 workloads for the 8 socket HT enabled case and I needed to set N to more than 20 in order to get the big performance gains. One thing that I thought of was to have N be bas

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-18 Thread Peter Zijlstra
On Wed, Jul 17, 2013 at 09:02:24PM -0700, Jason Low wrote: > I ran a few AIM7 workloads for the 8 socket HT enabled case and I needed > to set N to more than 20 in order to get the big performance gains. > > One thing that I thought of was to have N be based on how often idle > balance attempts d

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-17 Thread Jason Low
On Wed, 2013-07-17 at 20:01 +0200, Peter Zijlstra wrote: > On Wed, Jul 17, 2013 at 01:51:51PM -0400, Rik van Riel wrote: > > On 07/17/2013 12:18 PM, Peter Zijlstra wrote: > > > >So the way I see things is that the only way newidle balance can slow down > > >things is if it runs when we could have

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-17 Thread Jason Low
On Wed, 2013-07-17 at 20:01 +0200, Peter Zijlstra wrote: > On Wed, Jul 17, 2013 at 01:51:51PM -0400, Rik van Riel wrote: > > On 07/17/2013 12:18 PM, Peter Zijlstra wrote: > > > >So the way I see things is that the only way newidle balance can slow down > > >things is if it runs when we could have

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-17 Thread Peter Zijlstra
On Wed, Jul 17, 2013 at 01:51:51PM -0400, Rik van Riel wrote: > On 07/17/2013 12:18 PM, Peter Zijlstra wrote: > >So the way I see things is that the only way newidle balance can slow down > >things is if it runs when we could have ran something useful. > > Due to contention on the runqueue locks

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-17 Thread Rik van Riel
On 07/17/2013 12:18 PM, Peter Zijlstra wrote: On Wed, Jul 17, 2013 at 08:59:01AM -0700, Jason Low wrote: Do you think its worth a try to consider each newidle balance attempt as the total load_balance attempts until it is able to move a task, and then skip balancing within the domain if a CP

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-17 Thread Peter Zijlstra
On Wed, Jul 17, 2013 at 08:59:01AM -0700, Jason Low wrote: > > So if we have the following: > > for_each_domain(sd) > before = sched_clock_cpu > load_balance(sd) > after = sched_clock_cpu > idle_balance_completion_time = after - before > > At this point, the "idle_balanc

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-17 Thread Jason Low
Hi Peter, On Wed, 2013-07-17 at 11:39 +0200, Peter Zijlstra wrote: > On Wed, Jul 17, 2013 at 01:11:41AM -0700, Jason Low wrote: > > For the more complex model, are you suggesting that each completion time > > is the time it takes to complete 1 iteration of the for_each_domain() > > loop? > > Per

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-17 Thread Peter Zijlstra
On Wed, Jul 17, 2013 at 01:11:41AM -0700, Jason Low wrote: > For the more complex model, are you suggesting that each completion time > is the time it takes to complete 1 iteration of the for_each_domain() > loop? Per sd, yes? So higher domains (or lower depending on how you model the thing in yo

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-17 Thread Jason Low
On Wed, 2013-07-17 at 09:25 +0200, Peter Zijlstra wrote: > On Tue, Jul 16, 2013 at 03:48:01PM -0700, Jason Low wrote: > > On Tue, 2013-07-16 at 22:20 +0200, Peter Zijlstra wrote: > > > On Tue, Jul 16, 2013 at 12:21:03PM -0700, Jason Low wrote: > > > > When running benchmarks on an 8 socket 80 core

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-17 Thread Peter Zijlstra
On Wed, Jul 17, 2013 at 09:25:04AM +0200, Peter Zijlstra wrote: > One thing that I thought of since is that we need to consider what > happens for people with a low resolution sched_clock. IIRC there are > still platforms that are jiffy based. Ignore that, they're all UP. -- To unsubscribe from th

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-17 Thread Peter Zijlstra
On Tue, Jul 16, 2013 at 03:48:01PM -0700, Jason Low wrote: > On Tue, 2013-07-16 at 22:20 +0200, Peter Zijlstra wrote: > > On Tue, Jul 16, 2013 at 12:21:03PM -0700, Jason Low wrote: > > > When running benchmarks on an 8 socket 80 core machine with a 3.10 kernel, > > > there can be a lot of contentio

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-16 Thread Jason Low
On Tue, 2013-07-16 at 22:20 +0200, Peter Zijlstra wrote: > On Tue, Jul 16, 2013 at 12:21:03PM -0700, Jason Low wrote: > > When running benchmarks on an 8 socket 80 core machine with a 3.10 kernel, > > there can be a lot of contention in idle_balance() and related functions. > > On many AIM7 workloa

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-16 Thread Peter Zijlstra
On Tue, Jul 16, 2013 at 12:21:03PM -0700, Jason Low wrote: > When running benchmarks on an 8 socket 80 core machine with a 3.10 kernel, > there can be a lot of contention in idle_balance() and related functions. > On many AIM7 workloads in which CPUs go idle very often and idle balance > gets calle

Re: [RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-16 Thread Rik van Riel
On 07/16/2013 03:21 PM, Jason Low wrote: When running benchmarks on an 8 socket 80 core machine with a 3.10 kernel, there can be a lot of contention in idle_balance() and related functions. On many AIM7 workloads in which CPUs go idle very often and idle balance gets called a lot, it is actually

[RFC] sched: Limit idle_balance() when it is being used too frequently

2013-07-16 Thread Jason Low
When running benchmarks on an 8 socket 80 core machine with a 3.10 kernel, there can be a lot of contention in idle_balance() and related functions. On many AIM7 workloads in which CPUs go idle very often and idle balance gets called a lot, it is actually lowering performance. Since idle balance o