On 1/8/20 12:50 PM, Geoffrey Brown wrote:
Instead of changing the reviewers, how about:
- we remind the sheriffs to needinfo
- #intermittent-reviewers check that needinfo is in place when
reviewing disabling patches.
To try and help address the visibility issue, we could also make
searchfox consume the test-info-disabled-by-os taskcluster task[1] and:
- put banners at the top of test files that say "Hey! This is
(sometimes) disabled on [android/linux/mac/windows]"
- put collapsible sections at the top of the directory listings that
explicitly call out the disabled tests in that directory. The idea would
be to avoid people needing to scroll through the potentially long list
of files to know which are disabled and provide some ambient awareness
of disabled tests.
If there's a way to get a similarly pre-built[2] mapping that would
provide information about the orange factor of tests[3] or that it's
been marked as needswork, that could also potentially be surfaced.
Andrew
1:
https://searchfox.org/mozilla-central/rev/be7d1f2d52dd9474ca2df145190a817614c924e4/taskcluster/ci/source-test/file-metadata.yml#62
2: Emphasis on pre-built in the sense that searchfox's processing
pipeline really doesn't want to be issuing a bunch of dynamic REST
queries that would add to its processing latency. It would want a
taskcluster job that runs as part of the nightly build process so it can
fetch a JSON blob at full network speed.
3: I guess test-info-all at
https://searchfox.org/mozilla-central/rev/be7d1f2d52dd9474ca2df145190a817614c924e4/taskcluster/ci/source-test/file-metadata.yml#91
does provide the "failed runs" count and "total runs" which could
provide the orange factor? The "total run time, seconds" scaled by the
"total runs" would definitely be interesting to surface in searchfox.
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
