On 08/17/2011 04:59 PM, Michael Hope wrote: > On Wed, Aug 17, 2011 at 11:12 PM, Dave Martin <dave.mar...@linaro.org> wrote: >> On Tue, Aug 16, 2011 at 7:14 PM, Zach Pfeffer <zach.pfef...@linaro.org> >> wrote: >>> Nicolas, >>> >>> Thanks for the notes. As you say there are many, many things that can >>> affect this demo. What notes like this really underscore is the >>> importance of staying up-to-date. This demo is more about the >>> macroscopic effects from tip support than anything else. We do have >>> some more specific benchmark numbers at: >>> >>> https://wiki.linaro.org/Platform/Android/AndroidToolchainBenchmarking >> >> If we're confident that the benchmark produces results of a >> trustworthy quality, then that's fine. I don't know this benchmark in >> detail, so I can't really judge, other than that the results look a >> bit odd. > > Ditto on that. Have these benchmarks been qualified? Do they > represent real workloads? Where do they come from? What aspects of > the system (CPU, memory, I/O, kernel, SMP) do they exercise? How > sensitive are they to minor changes?
The benchmark code comes from Android: http://android.git.kernel.org/?p=toolchain/benchmark.git I'm not an expert on benchmarking. I've just tried to focus on running these in a way that's as fair and repeatable as possible. > gnugo in particular is a problem - the results don't change across a > range of toolchains which suggests it's got a silly hot loop or isn't > core bound. > _______________________________________________ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
