On 08/15/2017 04:29 PM, jma...@mozilla.com wrote:
I would propose running these above tests on windows7-opt (we don't have these
running yet in windows 10, although we are close), and only running specific
tests which are not run in e10s mode, turning them off December 29th, 2017.
Keep in mind we have had many years to get all our tests running in e10s mode
and we have known since last year that Firefox 57 would be the first release
that we ship e10s by default to our users- my proposal is a 4 month temporary
measure to let test owners finish ensuring their tests are running in e10s mode.
Hi Joel,
I think Ben is right on here. Please note that it's not just service
workers that have a different implementation in e10s vs. non-e10s, many
other super important parts of the browser are in the same boat. For
example, our entire HTTP(S) stack has different code paths in e10s vs.
non-e10s. Furthermore, even in places where we share a lot of code in
the two modes, there are often subtle differences in the behavior in
between the two modes. I hope this is convincing in terms of the
necessity of maintaining the test coverage across the two modes.
To slightly readjust what you stated above from the perspective of the
Firefox developers, we have known since last years that Firefox 57 would
be the first release that we ship e10s by default for our *desktop*
users, and we have also known that Firefox 57 for Android will be
shipped without e10s for our *mobile* users, so it shouldn't be any
surprise why no large scale effort has been put into porting all of our
tests to run into e10s mode. Also please note that some of the test
frameworks you have listed above like browser-chrome aren't relevant to
Android, and some like jsreftests aren't relevant to e10s, so we should
probably have a more detailed conversation about the remaining ones.
I think a better way to think about this problem is perhaps how to get
to a point where we never end up losing test coverage on code that we
ship on our tier 1 platforms. Thinking about this in terms of
date-based deadlines where we don't have the human power to do the work
necessary isn't particularly helpful, and would ultimately result us in
losing invaluable test coverage. If past history is any lesson for us,
regressions will creep in as a result, and especially due to the fact
that this is mostly affecting Firefox for Android where we have the
lowest pre-release testing population among our tier1 products (AFAIK)
makes that extremely risky.
Therefore, I think we should:
* start running all of the non-e10s tests that can affect Android
again immediately.
* work out a realistic plan between engineering and the automation
team to address the existing gaps that prevent us from turning off
non-e10s tests without losing coverage on Android, and execute that plan.
* turn non-e10s tests off when the above has been done.
Please let me know what you think!
Thanks,
Ehsan
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
