Hey Joshua,
That's awesome!
How long does the try run take that generated this data? We should
consider scheduling a periodic job to collect this data and track it
over time.
Jonathan
On 7/6/2014 10:02 PM, Joshua Cranmer 🐧 wrote:
I don't know how many people follow code-coverage updates in general,
but I've produced relatively up-to-date code coverage results based on
<http://hg.mozilla.org/mozilla-central/rev/81691a55e60f>, and they may
be found here: <http://www.tjhsst.edu/~jcranmer/m-ccov/>.
In contrast to earlier versions of my work, you can actually explore
the coverage as delineated by specific tests, as identified by their
TBPL identifier. Christian's persistent requests for me to limit the
depth of the treemap view are still unresolved, because, well, at 2 AM
in the morning, I just wanted to push a version that worked.
The test data was generated by pushing modified configs to try and
using blobber features to grab the resulting coverage data. Only
Linux32/64 is used, and only "opt" builds are represented (it's a
--disable-optimize --disable-debug kind of build), the latter because
I wanted to push a version out tonight and the debug .gcda tarballs
are taking way too long to finish downloading.
Effectively, only xpcshell tests, and the M, M-e10s, and R groups are
represented in the output data. M-e10s is slightly borked: only
M-e10s(1) [I think] is shown, because, well, treeherder didn't
distinguish between the five of them. A similar problem with the debug
M(dt1/dt2/dt3) test suites will arise when I incorporate that data.
C++ unit tests are not present because blobber doesn't run on C++ unit
tests for some reason, and Jit-tests, jetpack tests, and Marionette
tests await me hooking in the upload scripts to those testsuites (and
Jit-tests would suffer a similar numbering problems). The individual
testsuites within M-oth may be mislabeled because I can't sort names
properly.
There's a final, separate issue with treeherder not recording the
blobber upload artifacts for a few of the runs (e.g., Linux32 opt X),
even though it finished without errors and tbpl records those
artifacts. So coverage data is missing for the affected run. It's also
worth noting that a few test runs are mired with timeouts and
excessive failures, the worst culprit being Linux32 debug where half
the testsuites either had some failures or buildbot timeouts (and no
data at all).
If you want the underlying raw data (the .info files I prepare from
every individual run's info), I can provide that on request, but the
data is rather large (~2.5G uncompressed).
In short:
* I have up-to-date code-coverage on Linux 32-bit and Linux 64-bit.
Opt is up right now; debug will be uploaded hopefully within 24 hours.
* Per-test [TBPL run] level of detail is visible.
* Treeherder seems to be having a bit of an ontology issue...
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
