Sorry to bring back an old thread, but the upcoming "Test the web
forward" meeting reminded me of this thread.
In general I really approve of this idea, however I have one major concern.
> 2) Write an introduction to testharness.js targeted at people familiar
> with mochitest. testharness.js is the de facto standard testing
> harness in the web standards world, and we already can run such tests
> as mochitests automatically (see dom/imptests/), so JavaScript tests
> meant to be usable by other browsers should be written in that format.
As others have pointed out, the testharness.js test suite is much less
convenient to use than mochitest.
Simply wrapping
test(function() {
// test here
})
only works for the most simple tests. Most tests that I write use lots
of synchronous and asynchronous callbacks. Each one of those needs to
be wrapped to catch exceptions. There's also a lot more overhead in
the harness due to trying to count how much of a test you pass.
In general, testharness.js seems to be more optimized for producing a
result report which measure how close an implementation is to
implementing a feature, than it is optimized for making it easy to
write tests.
I believe many developers right now end up spending as much time
writing tests as they do implementing features. That is a very high
cost, but something that is definitely worth it. However we should be
working towards lowering that cost rather than increasing it.
Rather than trying to convince developers that testharness.js would in
fact not increase the cost of writing tests, I think we should try to
get W3C to adjust testharness.js such that it's easier to write tests
for it. If we make it as easy to write W3C tests as it is to write
mochitests then I would absolutely agree with your proposal. I would
imagine that would also make it easier to get other browser vendors to
do the same, as well as members of the webdevelopment community.
Another problem that I think we'd have is that many of our tests use
generators and yield. This *dramatically* cuts down on the complexity
of writing complex tests which has lots of asynchronous callbacks. For
example [1][2] would have been much harder to write without them. I
think our approach here could be migrate these tests to use ES6 based
generators as soon as we have them implemented in gecko, and then
submit them to W3C as soon as enough browsers implement ES6.
I don't think that we should be telling people to not use generators
in the meantime. My experience is that rewriting tests to use
generators both cuts down on the test writing time, and makes it much
less likely that the test ends up with intermittent orange bugs.
[1]
http://mxr.mozilla.org/mozilla-central/source/content/base/test/test_xhr_progressevents.html?force=1
[2]
http://mxr.mozilla.org/mozilla-central/source/dom/indexedDB/test/unit/test_add_put.js
/ Jonas
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
