James, thanks so much for the additional background information about testing at Mozilla.
I’m currently following the bugs you mentioned earlier and am looking forward to their results! Mike. On 03 Jun 2014, at 16:07, James Graham <ja...@hoppipolla.co.uk> wrote: > On 03/06/14 14:16, Mike de Boer wrote: > >> Writing wrappers in python around things to improve the current >> situation like a band-aid isn’t the way I’m used to fix things; I >> like to take the bull by the horns[1] >> >> I’d like to ask _why_ structured logging needs to be bolted on top of >> what we have currently? Is it more work to fix the real problem? Are >> we less comfortable doing things in JS? > > I'm not sure what "wrappers in python" you have in mind, but I think > that there are a couple of important points to address here. > > The first is a general point about testing at Mozilla. There is a lot > more going on than just testing of js code. We have tests written in, > and for, C++, Javascript, Python, HTML, CSS and probably a bunch more > things that I have forgotten. In terms of requirements it's pretty > different from what's needed to test a small project entirely > implemented in a single language. > > As it happens a lot of the orchestration of testing is implemented in > Python. I doubt js was even a viable choice at the time this > infrastructure was originally written, and now we have a set of mature > libraries for dealing with lots of the incidental complexity of testing > Mozilla products, like controlling system processes, and setting up B2G > devices. This code is largely decoupled from the tests themselves and > hopefully isn't something that most developers should need to care about. > > But the reason that we don't just throw it all away and start again is > because doing so would be a huge cost for extremely uncertain benefit. > That doesn't mean that we can't work to improve things where they are > bad; I'm sure we all have areas that we think are ripe for change. > > In terms of structured logging in particular; I don't know why you think > it's "bolted on", or why it isn't fixing the real problem. To be honest > I don't know what you think "the real problem" is. Structured logging is > basically just an API for communicating the results of tests to anyone > that cares to be notified of them, be it tbpl, mach, treeherder, an app > running in your browser that collects test results, or whatever. The > fact that it currently has a Python implementation just reflects the > fact that a lot of our test infrastructure is Python, and doesn't mean > that there's anything Python-specific about it. One could certainly > implement StructuredLog.jsm that would be entirely interoperable with > the Python code. Indeed, if you are writing a test harness that works > entirely in javascript, I would strongly encourage you to do just that. > The work on converting Mochitest to the new format might well have some > code that you could reuse. > > If there are particular requirements that you think structured logging > doesn't meet this is probably a good time to discuss them, since we > haven't deployed it to too many places. Perhaps that discussion would be > better off-list since it might not be of interest to everyone. > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
