On 06/26/2017 09:52 AM, Armen Zambrano Gasparnian wrote:
On Friday, 23 June 2017 10:05:42 UTC-4, Ehsan Akhgari wrote:
On Fri, Jun 23, 2017 at 4:19 AM, Chris Peterson <cpeter...@mozilla.com>
wrote:
On 6/23/17 12:17 AM, Ehsan Akhgari wrote:
But to speak of a more direct measurement of performance, let's look at
our progress on Speedometer V2
<https://github.com/WebKit/webkit/tree/master/PerformanceTests/Speedometer>.
Ehsan, were you comparing against http://speedometer2.benj.me?
I used this instance for these measurements.
Or this http://browserbench.org/Speedometer/?
This is still Speedometer V1, FWIW.
Or using the AWFY code? (It uses local proxy)
Today, I measured our progress so far on this benchmark by comparing
Firefox 53, 54, 55.0b3 (latest beta as of this writing) and the latest
Nightly, all x64 builds, on the reference hardware
<https://www.amazon.com/dp/B01K1IO3QW>. This is the result (numbers are
the reported benchmark score, higher is better):
[image: Speedometer improvements]
How do these Speedometer V2 scores map to the results on AWFY? AWFY shows
many Speedometer sub-tests, but no score in the range of 70.21. AWFY
machine #36 is the reference hardware.
https://arewefastyet.com/#machine=36&view=breakdown&suite=speedometer-misc
Armen has been investigating the difference. The Speedometer benchmark is
extremely sensitive to anything else that is going on on the machine at the
time you are running the tests, see for example
https://bugzilla.mozilla.org/show_bug.cgi?id=1373396#c3 where he discovered
that turning off the "Shut off display after 15 minutes" setting improves
our benchmark score by about 10 points or so! I think there are other
investigations ongoing to dig into the remaining difference as well.
Currently, machine #36 is testing against non-PGO builds from inbound (to catch
regressions).
I'm looking into adding mozilla-central PGO builds to the mix.
I assume PGO builds can run slightly faster than non-PGO builds.
Once we have PGO builds we will be able to see if there are anymore machine
configration changes required (I doubt it).
Direct link to the speedometer score on machine #36:
https://arewefastyet.com/#machine=36&view=single&suite=speedometer-misc&subtest=score
(Also note that there is a speed difference on Speedometer between Nightly
and Beta, where Nightly with the same code will be a bit slower than Beta
since some Nightly specific features do show up in Speedometer profiles
currently, for example things like bug 1375568 are currently Nightly only,
and there are also debugging checks like <
https://searchfox.org/mozilla-central/rev/3291398f10dcbe192fb52e74974b172616c018aa/ipc/chromium/src/base/pickle.h#26>
that show up a bit in profiles as well. I think that explains why 55.0b3
is scoring so high comparing to 56.0a1 there.)
Cheers,
--
Ehsan
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
