On 8/19/2014 2:41 PM, Ehsan Akhgari wrote:
On 2014-08-19, 3:57 PM, Jeff Gilbert wrote:
I would actually say that debug tests are more important for
continuous integration than opt tests. At least in code I deal with,
we have a ton of asserts to guarantee behavior, and we really want
test coverage with these via CI. If a test passes on debug, it should
almost certainly pass on opt, just faster. The opposite is not true.
"They take a long time and then break" is part of what I believe
caused us to not bother with debug testing on much of Android and
B2G, which we still haven't completely fixed. It should be
unacceptable to ship without CI on debug tests, but here we are
anyways. (This is finally nearly fixed, though there is still some
work to do)
I'm not saying running debug tests less often is on the same scale of
bad, but I would like to express my concerns about heading in that
direction.
I second this. I'm curious to know why you picked debug tests for
this experiment. Would it not make more sense to run opt tests on
desktop platforms on every other run?
Just based on the fact that they take longer and thus running them less
frequently would have a larger impact. If there's a broad consensus
that debug runs are more valuable, we could switch to running opt tests
less frequently instead.
Jonathan
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
