On 01/07/2014 08:59 PM, Peter Zijlstra wrote:
> On Tue, Jan 07, 2014 at 12:55:18PM +0000, Morten Rasmussen wrote:
>> My understanding is that should_we_balance() decides which cpu is
>> eligible for doing the load balancing for a given domain (and the
>> domains above). That is, only one cpu in a group is allowed to load
>> balance between the local group and other groups. That cpu would
>> therefore be reponsible for pulling enough load that the groups are
>> balanced even if it means temporarily overloading itself. The other cpus
>> in the group will take care of load balancing the extra load within the
>> local group later.
Thanks for both of you comments and explanations! :)
I know this patch's change is arguable and my attempt doesn't tune well. But I
believe I am in a correct way. :) let me explain a bit for this patch again.
First cpu_load includes the history load info, so repeatedly decay and use the
history load is kind of non-sense. and the old source/target_load randomly
select history load or current load just according to max/min, it also owe a
well explanation.
Second, we consider the bias in source/target_load already. but still use
imbalance_pct as last check in idlest/busiest group finding. It is also a kind
of redundant job. If we can consider the source/target bias, we'd better not
use imbalance_pct again.
And last, imbalance pct overused with quickly core number increasing cpu. Like
in find_busiset_group:
Assume a 2 groups domain, each group has 8 cores cpus.
The target group will bias 8 * (imbalance_pct -100)
= 8 * (125 - 100) = 200.
Since each of cpu bias .25 times load, for 8 cpus, totally bias 2 times
average cpu load between groups. That is a too much. But if there only 2 cores
in cpu group(common case when the code introduced). the bias is just 2 * 25 /
100 = 0.5 times average cpu load.
Now this patchset remove the cpu_load array avoid repeated history decay;
reorganize the imbalance_pct usage to avoid redundant balance bias. and reduce
the bias value between cpu groups -- maybe it isn't tune well. :)
>
> Correct.
>
>> I may have missed something, but I don't understand the reason for the
>> performance improvements that you are reporting. I see better numbers
>> for a few benchmarks, but I still don't understand why the code makes
>> sense after the cleanup. If we don't understand why it works, we cannot
>> be sure that it doesn't harm other benchmarks. There is always a chance
>> that we miss something but, IMHO, not having any idea to begin with
>> increases the chances for problems later significantly. So why not get
>> to the bottom of the problem of cleaning up cpu_load?
>>
>> Have you done more extensive benchmarking? Have you seen any regressions
>> in other benchmarks?
>
> I only remember hackbench numbers and that generally fares well with a
> more aggressive balancer since it has no actual work to speak of the
> migration penalty is very low and because there's a metric ton of tasks
> the aggressive leveling makes for more coherent 'throughput'.
I just tested hackbench on arm. and with more testing times plus rebase to
.13-rc6, the variation increased, then the benefit become unclear. anyway still
no regression find on both perf-stat cpu-migration times and real execute time.
On 0day performance testing should tested kbuild, hackbench, aim7, dbench,
tbench, sysbench, netperf etc. etc. No regression found.
The 0day performance testing also catch a cpu migration reduced on kvm guest.
https://lkml.org/lkml/2013/12/21/135
and another benchmark get benefit on the old patchset:
like the testing results show on 0day performance testing:
https://lkml.org/lkml/2013/12/4/102
Hi Alex,
We obsevered 150% performance gain with vm-scalability/300s-mmap-pread-seq
testcase with this patch applied. Here is a list of changes we got so far:
testbox : brickland
testcase: vm-scalability/300s-mmap-pread-seq
f1b6442c7dd12802e622 d70495ef86f397816d73
(parent commit) (this commit)
------------------------ ------------------------
26393249.80 +150.9% 66223933.60 vm-scalability.throughput
225.12 -49.9% 112.75 time.elapsed_time
36333.40 -90.7% 3392.20 vmstat.system.cs
2.40 +375.0% 11.40 vmstat.cpu.id
3770081.60 -97.7% 87673.40 time.major_page_faults
3975276.20 -97.0% 117409.60
time.voluntary_context_switches
3.05 +301.7% 12.24 iostat.cpu.idle
21118.41 -70.3% 6277.19 time.system_time
18.40 +130.4% 42.40 vmstat.cpu.us
77.00 -41.3% 45.20 vmstat.cpu.sy
47459.60 -31.3% 32592.20 vmstat.system.in
82435.40 -12.1% 72443.60
time.involuntary_context_switches
5128.13 +14.0% 5848.30 time.user_time
11656.20 -7.8% 10745.60
time.percent_of_cpu_this_job_got
1069997484.80 +0.3% 1073679919.00 time.minor_page_faults
Btw, the latest patchset include more clean up.
g...@github.com:alexshi/power-scheduling.git noload
Guess fengguang's 0day performance is doing test on it.
--
Thanks
Alex
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
