https://wpt.fyi is a dashboard containing the results of the full
web-platform-tests suite in multiple versions of current browsers.
The default view shows test results in the latest stable releases of
desktop Firefox/Chrome/Edge/Safari. There are several alternative views
that are likely to be of interest to gecko developers:
* The "label" url parameter can be used to change the browser channel.
https://wpt.fyi/?label=experimental shows the latest test results in
Firefox Nightly, Chrome Dev and latest Edge Insider / Safari TP.
label=beta gives results for Chrome and Firefox betas.
* The diff parameter in combination with others can be used to construct
a two-way comparison. For example
https://wpt.fyi/results/?products=firefox[beta],firefox[stable]&aligned=true&diff
shows Firefox beta vs stable, and
https://wpt.fyi/results/?products=chrome[experimental],firefox[experimental]&aligned=true&diff
shows a comparison between latest Firefox and Chrome. The diff
comparison is currently lossy as it shows the sum of changes, which can
misrepresent the case where /different/ tests pass and fail across the
two browsers; a solution to this is currently under development.
* The "Interoperability" view (link on the main page) is an attempt to
show which platform features have good/poor interop, as defined by the
fraction of tests that have the same result in multiple browsers.
UI to replace the current LBI ("location bar interface" :) is planned work.
If there are changes that you need to make wpt.fyi better meet your
needs please speak to me or file a bug at
https://github.com/web-platform-tests/wpt.fyi/issues
Thanks to our colleagues in the Chrome Platform Predictability team and
their associates at Bocoup who did the bulk of the work to make the
dashboard happen.
= (Possibly) FAQ =
How often are the results updated?
Experimental (i.e. nightly/dev) builds of Firefox and Chrome are run on
Linux using Taskcluster after each commit to web-platform-tests. The
test runs can be found on
https://github.com/web-platform-tests/wpt/commits/master (click on the
green tick). Beta builds are run weekly and stable builds daily. Safari
and stable Edge are run on a buildbot instance hosted by Bocoup and are
updated roughly daily (there are also buildbot runs of Firefox and
Chrome stable). Edge insider results are provided by Microsoft, sometimes.
What browser settings are used?
For Firefox we apply the same profile settings we use for internal CI
from testing/profiles (but not test-specific prefs set in
testing/web-platform/meta). For stable/beta builds the prefs are taken
from the user.js files in release/beta repositories. Note that this
means that non-experimental builds are run with whatever prefs are
enabled there. I'd like to clean this up so that beta/release more
closely match released beta/release. Chrome uses
--enable-experimental-web-platform-features for experimental builds and
only prefs required for the test harness to run in other configurations.
I believe that Edge and Safari are also run with minimal changes from
the default pref set.
How are the tests run?
Tests are run using the `wpt run` command in the web-platform-tests
repository. For example you can replicate the results from firefox beta
running something like `wpt run --channel beta --install-browser firefox
<path>`. The docker image used for Taskcluster runs is in tools/docker/.
The underlying harness is wptrunner which also used for Gecko CI.
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
