On Monday, January 11, 2016 at 8:42:11 AM UTC-8, William Lachance wrote: > It seems like another alternative might be to run Octane in Talos, > instead of v8_7. > > It seems like Talos has two advantages over AWFY (correct me if I'm wrong): > > 1. Easy for developers to schedule jobs via try (maybe less of a concern > with a benchmark like this, where I suspect results are more > reproducible locally?)
I believe there was talk of adding try support for AWFY (there already is for AWSY). Of course that's not actually done yet, I just want to point out it's not particularly hard and AWSY's version could be adapted rather easily. > 2. More hardware available, so can get results faster. I would guess we want to run on dedicated non-virtualized hardware for these tests. Is that an option w/ Talos? FWIW if that's an option I'd be more than happy to move AWSY over to the platform as well :) > Thoughts? Incidentally one of my deliverables for this quarter is to try > to figure out how Perfherder, Talos, and AWFY should co-exist, so I'm > very interested in knowing if my assumptions above are correct. Regardless of whether we use Talos to run the tests or not, it would be definitely be nice have the data reported in perfherder. A digression, maybe worth followup in a separate thread: In general it would be great if we could consolidate the various perf tests (AWFY, AWSY, Talos, Raptor, etc) under one umbrella (at least from an end user perspective). So you could go to trychooser and choose a "Perf" option that would have various subsets like: "JS Engine", "Memory Usage", "Layout Latency", "Mobile Launch Time", etc. If all of these systems reported their data to perfherder (and optionally elsewhere) we'd now have one centralized location where you can track perf regressions. As an end-user this is pretty great: The graphs look the same across systems, I only have to learn how to use one tool, I only have to learn how to interpret regressions in one system. -e _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
