On Sat, Aug 18, 2012 at 12:38 AM, Justin Dolske <dol...@mozilla.com> wrote: > Given the long history (shall I say "plague"?) of intermittent-orange in our > tree, I can't agree that this would be a non-issue or is easy to fix! [Nor > am I saying reusable tests are a bad idea -- just that it would seem wise to > ramp up over time.]
To be fair, the reason intermittent orange is such a headache for us is because our infrastructure for it is terrible. It all revolves around running tests in giant indivisible blocks that produce semi-formatted plaintext output, which is then parsed (using regex?) by various ad hoc tools, and huge amounts have to be done by hand. Random orange would be a drastically smaller problem for us if, e.g., we ran tests individually instead of in giant chunks, and automatically reran any failed test a few times to see if the failure is intermittent, and restarted a run in the middle if something made it crashed. Based on what James is saying, it sounds like Opera has a substantially more sophisticated system than we do (embarrassingly?). Anyway, one major goal of an open web is that users should have as many choices as possible for web browsers. That means we need to put special effort into making things as easy as possible for smaller browsers. So if Opera will definitely use our tests and other browsers might or might not, I think that's a good reason to go ahead. _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
