On 9/28/12 8:32 PM, Chris Pearce wrote:
This is indeed unfortunate. However I'd prefer to add more capacity to
our test infrastructure, rather than discourage developers from properly
testing before landing.
I think the concern is the definition of "properly".
If Try is hogging resources needed by Inbound, we should lower the
priority of Try.
Try is hogging resources needed by _Try_. The last try push I did,
asking for one test on one platform, waited for over a day before the
test even started running.
Developers should run tests locally for the modules affected by their
code changes before pushing to Try.
That's a completely unreasonable requirement for many modules. For
example, pretty much any change to the JS engine would require the vast
majority of our tests to be run, based on that criterion.
Single-platform builds certainly won't catch those pesky WinXP-only
browser-chrome oranges. But that's what mozilla-inbound is for.
Inbound is not for catching pesky WinXP-only failures. Try is.
Try never runs WinXP tests, to a first approximation. See above about
wait times.
So what if you have to wait overnight for a Try run to complete on all
platforms?
If only it were "overnight". We're into multi-day lag territory.
Few patches so urgently need to land that they can't wait
until tomorrow to land.
The point is that the patches that _do_ need to land urgently are
blocked on lack of resources because people are wasting too many cycles
with unnecesary try pushes.
Likewise, attempts to use try to diagnose and debug problems on
platforms one does not have access to are adversely affected by long lag
times on try.
I agree that people _should_ be testing patches on try, but it's not all
or nothing. If you're running every minor change through full tests on
try, then you're using up compute resources that other people can put to
better use running patches that really _need_ try time.
> I'd even go as far to suggest that we should *require* a green Try
> run before allowing people to land, for everything except "simple"
> changes.
I'd be interested in an estimate of how many hours of machine time per
day this would require, and a comparison to how many such hours we
actually have available...
-Boris
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
