> On Apr 1, 2019, at 11:54 AM, Jared Hirsch <6...@mozilla.com> wrote: > > This sounds great! I land features in the tree periodically, but infrequently > enough to forget lots of little details. It would really save time (mine and > the time of people I ping on IRC for help...) if our tooling simplified the > process of creating new tests. > > I don't have any historical context on the reasons why the bug number was > added to the template, but removing it seems reasonable to me, fwiw. > > One small feature request: could you add some basic usage documentation to > the firefox-source-docs as part of this patch? I'm sure I'll forget the right > sequence of commands in between uses ;-)
Glad to hear this would save time, makes me more sure I should push it forward. For now I was planning to update the existing test template docs at https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Mochitest#Test_templates to use the mach command, since that’s where a lot of related APIs are already documented. I’m open to the argument that this entire page should live on firefox-source-docs, but would like to decouple that from this command if possible :). Brian > Cheers, > > Jared > > > On Mon, Apr 1, 2019 at 11:36 AM Brian Grinstead <bgrinst...@mozilla.com> > wrote: > Based on my own experience and discussions with others, the workflow for > adding new mochitests isn't great. Commonly, it looks like: "copy/paste a > test in the same directory, add the new test to the relevant manifest file, > empty out the actual test bits, write your test". In my experience this is > prone to issues like forgetting to add the new test to the manifest, or not > fully replacing boilerplate like bug numbers from the copied test. > > There's a script in tree I was unaware of until last week called > gen_template.pl that's intended to help here, but it does leave a few issues > open: > > 1) It doesn't help with finding the manifest file and adding the new test to > it. > 2) The boilerplate it generates is outdated (for example, it sets > type="application/javascript" even in HTML documents, it doesn't include > add_task, etc). > 3) It supports only mochitest-chrome and mochitest-plain. > > Last week I prototyped a new mach command to fix (1) and (2), and expand (3) > to include browser-chrome mochitests. If it's helpful, it could be extended > to more test types as well. When you run the command it will create a file > with the appropriate boilerplate and add it to the manifest file (chrome.ini, > mochitest.ini, browser.ini depending on the type). This way you can > immediately run the test with `./mach mochitest`. It looks like this: > > ``` > # Chrome mochitests > ./mach addtest js/xpconnect/tests/chrome/test_chrome.html > ./mach addtest js/xpconnect/tests/chrome/test_chrome.xhtml > ./mach addtest js/xpconnect/tests/chrome/test_chrome.xul > > # Plain mochitests > ./mach addtest js/xpconnect/tests/mochitest/test_plain.html > ./mach addtest js/xpconnect/tests/mochitest/test_plain.xhtml > ./mach addtest js/xpconnect/tests/mochitest/test_plain.xul > > # Browser mochitests > ./mach addtest browser/base/content/test/alerts/browser_mochitest.js > ``` > > It's not landed because I’d like to get some feedback (see next paragraph), > but if you'd like to try it locally it can be applied from > https://phabricator.services.mozilla.com/D25482. > > This change modifies the existing templates used by gen_template.pl and > removes the perl script. I think the template changes are mostly > uncontroversial, but there is one notable change that I'd like feedback on > before landing. We currently add a bug number to new tests generated with the > script (in 4 places). For example, see "{BUGNUMBER}" in > https://searchfox.org/mozilla-central/source/testing/mochitest/static/chrome.template.txt. > My patch removes this. The reasoning is: > > 1) To tighten up the boilerplate and prevent the bug number from not being > updated in any or all of the spots when a test is created via copy/paste. > 2) Require less information up front when generating the test (in case you > are building a test before filing a bug, or have a test not associated with > any one particular bug). > > Links to bugs/comments/etc can be added in the test if they are relevant, but > I don't know that it's important enough to add another step in front of > getting a useful test case built. I did also consider adding a TODO comment > in the template to add a bug link (though in a single place instead of 4), > but not to require that information up front. > > If I don’t hear objections on this point I’ll work towards getting it landed > soon. > > Thanks, > Brian > > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
