I started to summarise the things I’d like to see in a JS unit test runner here[1]:
* mini-core. * Async support as a base. We’ve added `add_task()` methods where possible, but we haven’t made it a core feature of the legacy suites in use today. Generators yielding Promises are now possible, but I think we can do better. This leads to the following point... * It’d be nice if a suite would be extensible is such a way that it’d be possible to plug-in static analysis tooling to catch race conditions, run linters, common coding pattern validations, common coding error traps, etc. All this should be possible to add separately as modules, so we don’t end up with 3000+ LOC frameworks that’ll become harder and harder to maintain over time. * Pluggable reporters to spew out different type of logging formats. TAP, spec, XUnit, to (log)file, etc, all configurable. * Pluggable code coverage tooling * Allow to single out specific tests to aid in local debugging * Allow to disable specific tests without the need to comment them out, so they will be registered as ‘skipped’ (depending on the reporter) * Support pluggable JS debugger support. Perhaps by running the DevTools remote in a separate XUL window? * Pluggable environments; the runner should be able to adapt to any JS environment, be it XPCShell, Mochi/ Fx browser, b2g, etc. * Should be independent of the assertion style used. Assert.jsm could be bundled by default. * It should be covered 100% by tests itself, Inception-style. * It should provide hooks to setup a test suite, an individual test, to tear down an individual test and a test suite. * It‘d be nice to support multiple styles of writing tests: BDD, TDD and QUnit. I expect this’ll raise eyebrows for some and tears of joy for others… please all respond! Mike. [1] https://bugzilla.mozilla.org/show_bug.cgi?id=867742 On 03 Jun 2014, at 15:48, Gijs Kruitbosch <gijskruitbo...@gmail.com> wrote: > I don't think this is off-topic. I'm essentially asking: why did you focus on > this, and why in this way? More broadly, I'm asking what it is you're missing > from node/mocha. > > If you think that needs its own topic, feel free to fork the summary. > > In any case, discussions about the ease of use of our test framework(s) seem > eminently on-topic for m.d.platform. > > ~ Gijs > > On 03/06/2014 14:45, Mike de Boer wrote: >> I understand all that and I *think* you missed the header mentioning I was >> going off-topic… >> >> Mike. >> >> On 03 Jun 2014, at 15:39, Gijs Kruitbosch <gijskruitbo...@gmail.com> wrote: >> >>> On 03/06/2014 14:16, Mike de Boer wrote: >>>> Indeed, I’m used to the NodeJS/ Mocha flow of writing tests as fast, or >>>> even faster, as writing the implementation of a feature. I could group >>>> tests, isolate one, hook in a debugger at any point and more. This is >>>> something I miss while working on Fx and to be honest: I still crave for >>>> more than a year later. >>> >>> >>> So I'm not used to the "NodeJS/Mocha" 'flow' of writing tests. Can you >>> explain what the big difference there is? I find it hard to believe the >>> names of the assertion functions are the one big thing making tests take >>> much longer to write... >>> >>> Node is inherently single-threaded (unless you bolt on extra stuff), which >>> makes the runtime environment a lot more controlled than say, a web >>> browser. IME, the main issue in writing correct browser mochitest tests >>> (which is what I write 95% of the time) is focus, timeouts, and event >>> sequences (ensuring you wait for X before doing Y), OS differences (and >>> sometimes, try/staging vs. my local machine differences) and making it easy >>> and obvious (how) to trigger UI actions in a test, and the ease of >>> debugging (which has gotten better now that we have --break-on-failure and >>> --jsdebugger). >>> >>> That all seems to have preciously little to do with assertions, log output, >>> or TDD per se. In other words, if we want to make the experience of writing >>> tests easier, I don't think a unified assertion framework is where we >>> should be focusing our effort. >>> >>> Can you expand on why you think that that is such an important component >>> here, and how it compares with mocha/node? >>> >>> ~ Gijs >> > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
