Re: Switching Jetpack to use the runtests.py automation

2014-08-05 Thread Jonathan Griffin
If this only involves tiny changes to mochitest and it's ready, I'd go ahead and do that. I am interested in seeing what your requirements are, though, and figuring out if we could meet them later with a better architected solution, whether it's Marionette or something else. Mochitest is kind

Re: Switching Jetpack to use the runtests.py automation

2014-08-05 Thread Dave Townsend
On Mon, Aug 4, 2014 at 6:21 PM, Gregory Szorc wrote: > On 8/4/14, 10:39 AM, Dave Townsend wrote: > >> I've done a little investigation into marionette and I've found a few >> issues with it: >> >> Firstly it doesn't look like running marionette directly or through mach >> allows developers to sel

Re: Switching Jetpack to use the runtests.py automation

2014-08-04 Thread Gregory Szorc
On 8/4/14, 10:39 AM, Dave Townsend wrote: I've done a little investigation into marionette and I've found a few issues with it: Firstly it doesn't look like running marionette directly or through mach allows developers to select individual directories or test files to run, rather it is a one-sho

Re: Switching Jetpack to use the runtests.py automation

2014-08-04 Thread Dave Townsend
I've done a little investigation into marionette and I've found a few issues with it: Firstly it doesn't look like running marionette directly or through mach allows developers to select individual directories or test files to run, rather it is a one-shot affair. This is very inconvenient for deve

Re: Switching Jetpack to use the runtests.py automation

2014-07-15 Thread Henrik Skupin
Gregory Szorc wrote on 07/15/2014 09:04 PM: > On 7/15/14, 11:49 AM, Dave Townsend wrote: >> Since forever Jetpack tests in the Firefox trees have been run using our >> custom python CFX tool which is based on a fork of an ancient version of >> mozrunner. This causes us a number of problems. Keepin

Re: Switching Jetpack to use the runtests.py automation

2014-07-15 Thread Dave Townsend
On Tue, Jul 15, 2014 at 12:15 PM, Ted Mielczarek wrote: > On 7/15/2014 2:49 PM, Dave Townsend wrote: > > Since forever Jetpack tests in the Firefox trees have been run using our > > custom python CFX tool which is based on a fork of an ancient version of > > mozrunner. This causes us a number of

Re: Switching Jetpack to use the runtests.py automation

2014-07-15 Thread Ted Mielczarek
On 7/15/2014 2:49 PM, Dave Townsend wrote: > Since forever Jetpack tests in the Firefox trees have been run using our > custom python CFX tool which is based on a fork of an ancient version of > mozrunner. This causes us a number of problems. Keeping up with tree > visibility rules is hard. Some fe

Re: Switching Jetpack to use the runtests.py automation

2014-07-15 Thread Gregory Szorc
On 7/15/14, 11:49 AM, Dave Townsend wrote: Since forever Jetpack tests in the Firefox trees have been run using our custom python CFX tool which is based on a fork of an ancient version of mozrunner. This causes us a number of problems. Keeping up with tree visibility rules is hard. Some features

Re: Switching Jetpack to use the runtests.py automation

2014-07-15 Thread Bobby Holley
I think I speak for everyone who's debugged JP failures when I say: Huzzah! Thanks for doing this Mossop :-) On Tue, Jul 15, 2014 at 11:49 AM, Dave Townsend wrote: > Since forever Jetpack tests in the Firefox trees have been run using our > custom python CFX tool which is based on a fork of an

Switching Jetpack to use the runtests.py automation

2014-07-15 Thread Dave Townsend
Since forever Jetpack tests in the Firefox trees have been run using our custom python CFX tool which is based on a fork of an ancient version of mozrunner. This causes us a number of problems. Keeping up with tree visibility rules is hard. Some features from newer versions of mozrunner like crash