I don't think that having papercuts in the workflow for writing one type of test is the right way to nudge developers into writing another type. It also doesn't seem effective - otherwise people would be using the wpt-create tool to avoid jumping through hoops to add a mochitest.
That said, given there’s already a convention for this perhaps the tool as-is would be better named `./mach mochitest-create`. Based on Steve’s suggestion, if we did want a single API we could do something like: # Attempt to automatically determine the type of test (mochitest-chrome, xpcshell, wpt, etc) `./mach addtest path/to/test` # If you want to pass extra arguments specific to that type, then you use a subcommand: `./mach addtest mochitest --flavor=chrome toolkit/components/windowcreator/test/test_chrome.html` `./mach addtest wpt testing/web-platform/tests/accelerometer/test.html --long-timeout` Brian > On Apr 2, 2019, at 2:40 AM, James Graham <ja...@hoppipolla.co.uk> wrote: > > On 01/04/2019 23:13, Steve Fink wrote: >> On 4/1/19 11:36 AM, Brian Grinstead wrote: >>> Based on my own experience and discussions with others, the workflow for >>> adding new mochitests isn't great. Commonly, it looks like: "copy/paste a >>> test in the same directory, add the new test to the relevant manifest file, >>> empty out the actual test bits, write your test". In my experience this is >>> prone to issues like forgetting to add the new test to the manifest, or not >>> fully replacing boilerplate like bug numbers from the copied test. >>> >>> There's a script in tree I was unaware of until last week called >>> gen_template.pl that's intended to help here, but it does leave a few >>> issues open: >>> >>> 1) It doesn't help with finding the manifest file and adding the new test >>> to it. >>> 2) The boilerplate it generates is outdated (for example, it sets >>> type="application/javascript" even in HTML documents, it doesn't include >>> add_task, etc). >>> 3) It supports only mochitest-chrome and mochitest-plain. >>> >>> Last week I prototyped a new mach command to fix (1) and (2), and expand >>> (3) to include browser-chrome mochitests. If it's helpful, it could be >>> extended to more test types as well. When you run the command it will >>> create a file with the appropriate boilerplate and add it to the manifest >>> file (chrome.ini, mochitest.ini, browser.ini depending on the type). This >>> way you can immediately run the test with `./mach mochitest`. >> It sounds great to me, but I'm wondering if the generic name is intentional >> or not. Various groups within Mozilla assume different things by 'test'. >> Is`mach addtest` intended to only be for mochitests? If so, then perhaps >> `mach addmochitest` is a better name, even it's a bit of mouthful. My >> reasoning is that there's already a distinction between `mach mochitest` and >> `mach test`, where the latter attempts to be general and support a bunch of >> different kinds of tests. Having `mach test` assume mochitests would be >> highly confusing to me, at least. (Though I'm not sure that `mach test` >> really works; it seems like I usually have to run the more specific command.) > > I'm also pretty worried by this. For web-platform features ensuring interop > is critical and as such web-platform-tests should be preferred over > mochitests where possible. But every time we build features with a > mochitest-first approach it undermines that. > > For web-platform-tests we already have ./mach wpt-create, so I think we > should either roll that functionality in to the new command as part of the > initial featureset or have one command per supported test type (i.e. call > this mach mochitest-create). > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
