> (Can you hear that thud, thud, thud? It's the sound of me beating my head > against my desk.)
One of the intriguing things about this benchmark is that it's open source, and they're committed to changing it over time. FWIW Paul Irish agrees the sieve is a bad test, although he doesn't hate it to the extent you or i would think is deserved. https://github.com/robohornet/robohornet/issues/20#issuecomment-8837867 So maybe all hope is not lost. It's really laughable that they count the sieve as a test of handlebars.js performance. Instead of, you know, actually using handlebars.js. But I again wonder how much we can mold this into something interesting. -Justin On Tue, Sep 25, 2012 at 1:41 AM, Nicholas Nethercote <n.netherc...@gmail.com> wrote: > On Mon, Sep 24, 2012 at 5:22 PM, Tim <timvk...@gmail.com> wrote: >> So there's a new benchmark out, seemingly from google. >> >> It is designed to test performance in web app bottlenecks, especially "DOM, >> <canvas> API methods, SVG". >> >> Paul Irish from Google's Chrome team is in charge of it. He blogged on it >> here: >> >> http://paulirish.com/2012/a-browser-benchmark-that-has-your-back-robohornet/ > > I'm horrified by this. Quoting my Hacker News comments > (http://news.ycombinator.com/item?id=4567796): > >> Oh god, just when web people were starting to understand how to create good >> benchmarks (https://blog.mozilla.org/nnethercote/2012/08/24/octane-minus...), >> now we're going back to 1980s microbenchmark hell. >> >> Doesn't anyone read Hennessy and Patterson any more? The best benchmarks >> are real apps, not crappy little microbenchmarks that measure a single thing. >> >> (Can you hear that thud, thud, thud? It's the sound of me beating my head >> against my desk.) > > Also, one of the tests is basically a no-op executed many times > (https://bugzilla.mozilla.org/show_bug.cgi?id=793913#c7). > > Even better, there's a prime numbers calculation test, apparently to > test "math". This is grimly hilarious: Hennessy and Patterson > specifically cite the Sieve of Erastosthenes as an example of a toy > (and thus crap) benchmark. Sigh. > > Daniel Buchner is apparently Mozilla's official representative on the > RoboHornet "committee of JavaScript experts" > (https://github.com/robohornet/robohornet/wiki/Committee-Membership). > I don't know what his role is, but the thought of Mozilla officially > blessing RoboHornet fills me with dread. > > While the suite may push us into some useful improvements, I worry > that we'll end up implementing some stupid benchmarketing features > that we will then carefully have to avoid regressing for the next 10 > years. > > Nick > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
