Based on my own experience and discussions with others, the workflow for adding new mochitests isn't great. Commonly, it looks like: "copy/paste a test in the same directory, add the new test to the relevant manifest file, empty out the actual test bits, write your test". In my experience this is prone to issues like forgetting to add the new test to the manifest, or not fully replacing boilerplate like bug numbers from the copied test.
There's a script in tree I was unaware of until last week called gen_template.pl that's intended to help here, but it does leave a few issues open: 1) It doesn't help with finding the manifest file and adding the new test to it. 2) The boilerplate it generates is outdated (for example, it sets type="application/javascript" even in HTML documents, it doesn't include add_task, etc). 3) It supports only mochitest-chrome and mochitest-plain. Last week I prototyped a new mach command to fix (1) and (2), and expand (3) to include browser-chrome mochitests. If it's helpful, it could be extended to more test types as well. When you run the command it will create a file with the appropriate boilerplate and add it to the manifest file (chrome.ini, mochitest.ini, browser.ini depending on the type). This way you can immediately run the test with `./mach mochitest`. It looks like this: ``` # Chrome mochitests ./mach addtest js/xpconnect/tests/chrome/test_chrome.html ./mach addtest js/xpconnect/tests/chrome/test_chrome.xhtml ./mach addtest js/xpconnect/tests/chrome/test_chrome.xul # Plain mochitests ./mach addtest js/xpconnect/tests/mochitest/test_plain.html ./mach addtest js/xpconnect/tests/mochitest/test_plain.xhtml ./mach addtest js/xpconnect/tests/mochitest/test_plain.xul # Browser mochitests ./mach addtest browser/base/content/test/alerts/browser_mochitest.js ``` It's not landed because I’d like to get some feedback (see next paragraph), but if you'd like to try it locally it can be applied from https://phabricator.services.mozilla.com/D25482. This change modifies the existing templates used by gen_template.pl and removes the perl script. I think the template changes are mostly uncontroversial, but there is one notable change that I'd like feedback on before landing. We currently add a bug number to new tests generated with the script (in 4 places). For example, see "{BUGNUMBER}" in https://searchfox.org/mozilla-central/source/testing/mochitest/static/chrome.template.txt. My patch removes this. The reasoning is: 1) To tighten up the boilerplate and prevent the bug number from not being updated in any or all of the spots when a test is created via copy/paste. 2) Require less information up front when generating the test (in case you are building a test before filing a bug, or have a test not associated with any one particular bug). Links to bugs/comments/etc can be added in the test if they are relevant, but I don't know that it's important enough to add another step in front of getting a useful test case built. I did also consider adding a TODO comment in the template to add a bug link (though in a single place instead of 4), but not to require that information up front. If I don’t hear objections on this point I’ll work towards getting it landed soon. Thanks, Brian _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
