Hi all
Last night bug 891882 landed, which broke the gaia ui tests for B2G
desktop on TBPL on b2g-inbound. This was backed out as a result, but
then relanded by the patch author, since they said the failures were
expected until they had landed the gaia counterpart. The gaia tests
counterpart wasn't being landed at the time, since it was waiting on a
green travis run, which required the gecko changes to have merged to
mozilla-central (travis uses the m-c builds).
This was suboptimal for a few reasons:
(To be clear, I don't blame the assignee of that bug - it isn't the
first time this scenario has occurred - we just need to agree on a
workflow and document/announce it, so as to avoid these problems in the
future)
1) Bug 891882 landed knowing it would break the TBPL tests, but the
sheriffs were not notified at the time, nor asked beforehand as to what
the preferred approach was (a comment in-bug when sheriffs aren't CCed
isn't clear enough IMO - sheriffs have to respond to failures asap and
as such don't always have time to read through a bug's comments). This
resulted in b2g-inbound being broken for 24 hours, until I backed out
bug 891882 again this morning.
2) Even with sufficient sheriff notification, the approach of "land on
b2g-inbound, wait for merge to mozilla-central, wait for travis green,
and only then land the gaia changes" is suboptimal since:
a) Sheriffs only merge trees that are not broken by default, and even
if we make an exception, we'll now have both b2g-inbound and
mozilla-central broken until the whole merge/travis/auto-sync dance is
complete - and mozilla-central is meant to be kept pristine.
b) A perma-failing test suite is at best yet another thing for
sheriffs/devs to keep track of (for the day or so this cycle takes to
complete) - and at worst, might cause us to miss further failures in
that test "since it's already red anyway".
c) We have no guarantee that the test fixes will actually work, so we
could end up with both mozilla-central and b2g-inbound still being
broken after 24+ hours.
Alternative workflows to the above are:
a) Land the gaia change even if the travis run isn't green, so that
TBPL is back to being green asap (and [2c] above can be ruled out).
b) [My preference]: First make a gaia change to disable the tests (or
make them feature-detect so they are backwards-compatible), then once
that's on mozilla-central, land the gecko change on b2g-inbound (since
the travis run will be green), then once that is on mozilla-central land
another gaia change to remove the support for the previous API/...
Unless anyone has any other ideas?
Also - what do we think about getting Travis to use the b2g-inbound
builds rather than mozilla-central? (In order to reduce the cycle time
when needing to make gecko changes)
Best wishes,
Ed
_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g
