On 8/20/14 10:58 AM, James Graham wrote:
On 20/08/14 18:38, Joshua Cranmer 🐧 wrote:
On 8/20/2014 12:22 PM, L. David Baron wrote:
(I estimated that it was going to be faster to get that working than
to try to figure out how to use the packaged tests, since it was
possible to reverse-engineer from mochitest run inside mach, though
if there had been instructions on how to use packaged tests that
somebody had actually used before I'd likely have gone the other
way.)
Building packaged tests is easy (make package for the installer, make
package-tests for the tests); running them is a little harder since you
have to build the python runtests.py command line yourself. Or you can
open up a tbpl log and grab the exact command line there. Certainly far
easier than trying to work out how to run mozharness on a local system...
Running mozharness on a local system is actually documented [1],
although I think that makes it sound harder than it actually is. I have
a run.sh script in the root of my mozharness checkout that looks like
#!/bin/bash
cd scripts
CONFIG_FILE=../configs/web_platform_tests/test_config.py
BUILD_ROOT=/home/jgraham/develop/gecko-dev/obj-x86_64-unknown-linux-gnu/dist
INSTALLER_URL=file://$BUILD_ROOT/firefox-34.0a1.en-US.linux-x86_64.tar.bz2
TEST_URL=file://$BUILD_ROOT/firefox-34.0a1.en-US.linux-x86_64.tests.zip
./web_platform_tests.py --no-read-buildbot-config --config-file
$CONFIG_FILE --installer-url $INSTALLER_URL --test-url $TEST_URL
Obviously for different testsuites you need different config files or
command lines, e.g. mochitest-plain would have something like
CONFIG_FILE=../configs/unittests/linux_unittest.py
and a final command like:
./desktop_unittest.py --no-read-buildbot-config --mochitest-suite plain
--config-file $CONFIG_FILE --installer-url $INSTALLER_URL --test-url
$TEST_URL --download-symbols true
[1]
https://wiki.mozilla.org/ReleaseEngineering/Mozharness/How_to_run_tests_as_a_developer
It sounds like the "copy build to remote machine so we can run tests
there" is somewhat common and needs to be better supported by the tools.
I think it makes sense to leverage test archives and build packages for
this. Since mozharness already supports running things from there, I
guess the missing piece is some glue code that invokes mozharness with a
sane set of default options. That missing piece sounds a lot like what
mach does. We talked about bundling mach in the tests archive to
facilitate things like this. Perhaps we should move forward with that?
Or is there a better way?
The problem is at the unholy intersection of build, test, and automation
responsibilities. Who's the owner? A-Team?
_______________________________________________
dev-platform mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-platform
