This looks awesome, Joshua! Thanks for doing this. CCing Sylvestre who
has also started to look into this...
Cheers,
Ehsan
On 2014-07-07, 1:02 AM, Joshua Cranmer 🐧 wrote:
I don't know how many people follow code-coverage updates in general,
but I've produced relatively up-to-date code coverage results based on
<http://hg.mozilla.org/mozilla-central/rev/81691a55e60f>, and they may
be found here: <http://www.tjhsst.edu/~jcranmer/m-ccov/>.
In contrast to earlier versions of my work, you can actually explore the
coverage as delineated by specific tests, as identified by their TBPL
identifier. Christian's persistent requests for me to limit the depth of
the treemap view are still unresolved, because, well, at 2 AM in the
morning, I just wanted to push a version that worked.
The test data was generated by pushing modified configs to try and using
blobber features to grab the resulting coverage data. Only Linux32/64 is
used, and only "opt" builds are represented (it's a --disable-optimize
--disable-debug kind of build), the latter because I wanted to push a
version out tonight and the debug .gcda tarballs are taking way too long
to finish downloading.
Effectively, only xpcshell tests, and the M, M-e10s, and R groups are
represented in the output data. M-e10s is slightly borked: only
M-e10s(1) [I think] is shown, because, well, treeherder didn't
distinguish between the five of them. A similar problem with the debug
M(dt1/dt2/dt3) test suites will arise when I incorporate that data. C++
unit tests are not present because blobber doesn't run on C++ unit tests
for some reason, and Jit-tests, jetpack tests, and Marionette tests
await me hooking in the upload scripts to those testsuites (and
Jit-tests would suffer a similar numbering problems). The individual
testsuites within M-oth may be mislabeled because I can't sort names
properly.
There's a final, separate issue with treeherder not recording the
blobber upload artifacts for a few of the runs (e.g., Linux32 opt X),
even though it finished without errors and tbpl records those artifacts.
So coverage data is missing for the affected run. It's also worth noting
that a few test runs are mired with timeouts and excessive failures, the
worst culprit being Linux32 debug where half the testsuites either had
some failures or buildbot timeouts (and no data at all).
If you want the underlying raw data (the .info files I prepare from
every individual run's info), I can provide that on request, but the
data is rather large (~2.5G uncompressed).
In short:
* I have up-to-date code-coverage on Linux 32-bit and Linux 64-bit. Opt
is up right now; debug will be uploaded hopefully within 24 hours.
* Per-test [TBPL run] level of detail is visible.
* Treeherder seems to be having a bit of an ontology issue...
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
