I think I speak for everyone who's debugged JP failures when I say: Huzzah!
Thanks for doing this Mossop :-) On Tue, Jul 15, 2014 at 11:49 AM, Dave Townsend <dtowns...@oxymoronical.com> wrote: > Since forever Jetpack tests in the Firefox trees have been run using our > custom python CFX tool which is based on a fork of an ancient version of > mozrunner. This causes us a number of problems. Keeping up with tree > visibility rules is hard. Some features from newer versions of mozrunner > like crash stack handling aren't available and our attempts to update to > the newer mozbase have been blocked on trying to get some of our forked > code accepted. It also makes it hard for Mozilla other developers to run > our tests as CFX has a very different syntax to the other test suites. > > We've started investigating switching away from CFX and instead using the > python automation that the mochitests use. This would work somewhat > similarly to browser-chrome tests, runtests.py will startup Firefox and > overlay some XUL and JS on the main window from where we can run the > existing JS parts of the Jetpack test suites. > > There are many benefits here. The runtests.py code is well used and known > to be resilient. It supports things like screenshots on failures and crash > stacks that Jetpack tests don't currently handle. We'll use manifest files > like the other test suites so disabling tests per platform will be easy. > Excellent mach integration will make running individual tests simple. It > also makes it possible to use commonjs style tests elsewhere in the tree. > Release engineering should find managing the Jetpack tests a lot easier as > they behave just like other mochitests. > > My initial experiment last week shows that this will work. The first part > of our tests (package tests) is running and passing on my local machine and > I expect to have the add-on tests working this week. > > I wanted to give everyone a heads up about this work to give you all a > chance to ask questions or raise objections. The changes to runtests and > the build system are minimal, just adding support for new manifest types > really but I will be needing reviews for those. We'll also have to make the > buildbot changes to switch over to use these new tests but I expect that to > be pretty straightforward. > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
