On 1/21/14, 10:25 AM, Ehsan Akhgari wrote:
On 1/21/2014, 12:54 PM, Gregory Szorc wrote:
On 1/21/14, 9:05 AM, Ehsan Akhgari wrote:
It seems to me like the splitting algorithm for mochitests gives us
different chunking results on different platforms, see this test failure
for example: <
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=746018b05d67>. Is
this
expected? If not, is this easy to fix so that a given test always falls
inside the same chunk if the number of chunks are equal on two different
platforms?
It's a hard problem until all tests are in .ini manifests and automation
is manifest driven.
Currently, the tests archive only contains mochitests that are active in
the current build configuration and the test runners look at the
filesystem for test discovery. To have consistent bucketing, we'd need
test archives to be the same on all configurations so the bucketing
algorithm would operate consistently [by taking skipped tests into
consideration].
Hmm, do you know what the current algorithm tries to do? I think that
we should be able to apply the same bucketing rule to all platforms, and
get buckets of potentially different sizes due to running different sets
of tests on different platforms, but still have each test appear in the
same test suite.
See testing/mochitest/chunkifyTests.js. AFAICT it's a simple bucket
grouping or a directory-based bucket grouping.
> Alternatively, we write the bucketing information into
the test archive at archive generation time. Either way, we need bug
920185 to be finished.
How close is that bug? There seems to be only two remaining blockers.
Are there blockers which have not been filed yet?
A very crude grep reveals 41 Makefile.in with "MOCHITEST" appearing in
them. More bugs likely need to be filed.
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
