On Tuesday, 25 September 2012 17:42:00 UTC+12, Nicholas Nethercote wrote: > On Mon, Sep 24, 2012 at 5:22 PM, wrote: > > > So there's a new benchmark out, seemingly from google. > > > > > > It is designed to test performance in web app bottlenecks, especially "DOM, > > <canvas> API methods, SVG". > > > > > > Paul Irish from Google's Chrome team is in charge of it. He blogged on it > > here: > > > > > > http://paulirish.com/2012/a-browser-benchmark-that-has-your-back-robohornet/ > > > > I'm horrified by this. Quoting my Hacker News comments > > (http://news.ycombinator.com/item?id=4567796): > > > > > Oh god, just when web people were starting to understand how to create good > > > benchmarks > > (https://blog.mozilla.org/nnethercote/2012/08/24/octane-minus...), > > > now we're going back to 1980s microbenchmark hell. > > > > > > Doesn't anyone read Hennessy and Patterson any more? The best benchmarks > > > are real apps, not crappy little microbenchmarks that measure a single > > thing. > > > > > > (Can you hear that thud, thud, thud? It's the sound of me beating my head > > > against my desk.) > > > > Also, one of the tests is basically a no-op executed many times > > (https://bugzilla.mozilla.org/show_bug.cgi?id=793913#c7). > > > > Even better, there's a prime numbers calculation test, apparently to > > test "math". This is grimly hilarious: Hennessy and Patterson > > specifically cite the Sieve of Erastosthenes as an example of a toy > > (and thus crap) benchmark. Sigh. > > > > Daniel Buchner is apparently Mozilla's official representative on the > > RoboHornet "committee of JavaScript experts" > > (https://github.com/robohornet/robohornet/wiki/Committee-Membership). > > I don't know what his role is, but the thought of Mozilla officially > > blessing RoboHornet fills me with dread. > > > > While the suite may push us into some useful improvements, I worry > > that we'll end up implementing some stupid benchmarketing features > > that we will then carefully have to avoid regressing for the next 10 > > years. > > > > Nick
The primes calculation inclusion is particularly odd. Especially when there are so many existing (useful) computation/math perf benchmarks. The original bug requesting the canvas clearing test is here: https://github.com/robohornet/robohornet/issues/43 _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
