On 1/21/2014, 2:19 PM, Gregory Szorc wrote:
On 1/21/14, 10:25 AM, Ehsan Akhgari wrote:
On 1/21/2014, 12:54 PM, Gregory Szorc wrote:
On 1/21/14, 9:05 AM, Ehsan Akhgari wrote:
It seems to me like the splitting algorithm for mochitests gives us
different chunking results on different platforms, see this test
failure
for example: <
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=746018b05d67>. Is
this
expected? If not, is this easy to fix so that a given test always
falls
inside the same chunk if the number of chunks are equal on two
different
platforms?
It's a hard problem until all tests are in .ini manifests and automation
is manifest driven.
Currently, the tests archive only contains mochitests that are active in
the current build configuration and the test runners look at the
filesystem for test discovery. To have consistent bucketing, we'd need
test archives to be the same on all configurations so the bucketing
algorithm would operate consistently [by taking skipped tests into
consideration].
Hmm, do you know what the current algorithm tries to do? I think that
we should be able to apply the same bucketing rule to all platforms, and
get buckets of potentially different sizes due to running different sets
of tests on different platforms, but still have each test appear in the
same test suite.
See testing/mochitest/chunkifyTests.js. AFAICT it's a simple bucket
grouping or a directory-based bucket grouping.
Right. OK, so the only way that the chunk would differ per platform
would be if an entire test directory is omitted on a given platform. I
guess it's possible to handle that would be to get a list of such
directories and special case them, but that doesn't seem like something
which I'm inclined to work on. So I guess we'll have to live with this
sub-optimal behavior for now. :(
Cheers,
Ehsan
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
