The upcoming changes sound great! Is there currently a way (or plans to add a way) to track regressions / improvements for a single measurement within a test? I see that in perfherder I can add these measurements to a graph (http://mzl.la/1E17Zyo <http://mzl.la/1E17Zyo>) but it’s hard to distinguish between normal variation across runs and an actual regression by looking at the graph.
Brian > On May 1, 2015, at 9:40 AM, jma...@mozilla.com wrote: > > It is always hard to advertise every change, but there are enough changes to > post a brief summary. > > What has changed: > 1) :bgrins has added a new talos test 'damp' - devtools at maximum > performance! This was done in > https://bugzilla.mozilla.org/show_bug.cgi?id=1150215 > 2) Talos doesn't run on OSX 10.6 anymore thanks to > https://bugzilla.mozilla.org/show_bug.cgi?id=990490 > 3) Talos runs on OSX 10.10 on main branches and is slowly replacing OSX 10.8 > out with uplifts on older branches. > 4) A few android tests were turned off, now we run 3 of them- ones that are > useful for tracking performance > > What changes are coming up: > 1) We are planning on reducing the pages we test for android tests, this is > mostly because the raw data is nearly identical to other pages. For > reference, this would reduce tp4m in half and tsvgx by a few pages. This > work will help us reduce over load as we transition from panda boards to real > devices this summer. > 2) compare-talos is being integrated into perfherder- This will be detailed > in a blog post with examples of how to use it for investigating regressions > and fixes. This work has been taking place in > https://bugzilla.mozilla.org/show_bug.cgi?id=1142680#c50 > 3) This quarter we will stop reporting to datazilla, this leaves graph server > and perfherder (the performance view inside of treeherder). > 4) talos counters will be reviewed and cleaned up this quarter as part of > https://bugzilla.mozilla.org/show_bug.cgi?id=1156907 > 5) we will start having compare-talos style views in bugs we file for > performance regressions > (https://bugzilla.mozilla.org/show_bug.cgi?id=1150616). > > All feedback is welcome. With active work on new tools and deliverables to > make things better, you can expect more changes. Of course, the more ideas > and feedback we get, the better !! > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
