tl;dr We now have a --verbose-if-fails option that outputs a log if a
xpcshell-test fails when being run in parallel.
If you ever have seen output from xpcshell-tests like this:
0:05.68 TEST_START:
toolkit/components/search/tests/xpcshell/test_remove_profile_engine.js
0:08.57 TEST_END: FAIL
0:08.61 INFO
toolkit/components/search/tests/xpcshell/test_remove_profile_engine.js failed
or timed out, will retry.
...
0:19.50 INFO Retrying tests that failed when run in parallel.
0:19.50 TEST_START:
toolkit/components/search/tests/xpcshell/test_remove_profile_engine.js
0:21.27 TEST_END: PASS
and wondered if that's an intermittent you're trying to fix or just why
is that failing... now you have a new tool.
--verbose is typically too noisy as it outputs all the logs of the
parallel tests at the same time.
I've just added the new option --verbose-if-fails which will output logs
for any failing test(s) during the parallel run.
0:07.72 TEST_END: FAIL
0:07.75 INFO >>>>>>>
0:06.49 INFO (xpcshell/head.js) | test MAIN run_test pending (1)
0:06.49 INFO (xpcshell/head.js) | test run_next_test 0 pending (2)
...More logs...
0:07.79 INFO <<<<<<<
0:07.79 INFO
toolkit/components/search/tests/xpcshell/test_require_engines_in_cache.js
failed or timed out, will retry.
...
I've used this style of output in the past to figure out why some of our
search tests were taking a long time to timeout, then would pass on the
second run. This is just formalising it into a usable option.
Mark.
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
