Justin, let's take a step back and not make invalid assertions.
* Constant fighting about "risk". Essentially all important patches
have to go through two layers of approval (review, blocking+).
Non-blocking patches have to go through three layers of approval
(review, tracking+, approval+) these days.
This is false. Any bug can request approval without being tracking+ (same as
desktop). Tracking nomination are a good way to understand whether the user
impact is enough to warrant uplift, or to get automatic uplift in the next
release.
In most cases, the risk assessment is completely pro-forma. Engineers
get what they want, in the end. But the process takes up time and
energy.
If it's not worth the 2 minutes for an engineer to request approval, it's
likely not worth uplifting the code change. And if approved, the bug is
uplifted automatically by sheriffs.
You're right though - you all could lie to us about risk or reward. We're
trusting you aren't.
I contend we would /never/ put up with this sort of process
BS in normal Firefox development.
This is so apples and oranges it's astonishing. How can you compare the landing
requirements of desktop releases (which involve no partners, have massive
pre-release testing populations, is a mature product, and has no hard feature
scope) to B2G releases?
-Alex
On Apr 3, 2013, at 1:27 PM, Justin Lebar <[email protected]> wrote:
If the past months have taught me anything, it's that "let's just
branch and double-land everything" is unfortunately not a path to
shipping quality software on time with happy, productive engineers.
So this is not a one-sided trade-off.
I get that there are partner constraints here, and perhaps riding the
trains is incompatible with them, but I think there's a certain amount
of creativity called for here, given that what we've been doing hasn't
been working. I'd be curious to know if you have any ideas in this
respect beyond "wait a few years".
I don't think there are any silver bullets I can offer. What exactly is the
problem we are trying to solve here though. How often do people still have to
double-land themselves? The feedback I was getting is that the vast majority of
landings is done by our awesome uplifting mini-team.
Even with our wonderful sheriff team, the many branches we have are a
huge pain. There are a lot of reasons. Here are some.
* Constant fighting about "risk". Essentially all important patches
have to go through two layers of approval (review, blocking+).
Non-blocking patches have to go through three layers of approval
(review, tracking+, approval+) these days.
In most cases, the risk assessment is completely pro-forma. Engineers
get what they want, in the end. But the process takes up time and
energy. I contend we would /never/ put up with this sort of process
BS in normal Firefox development.
* Divergence between the branches makes landing hard. It's not at all
uncommon for my patches to work on trunk and cause test failures on
b2g18. And this problem will only get worse the longer we keep these
branches around.
* Divergence between the branches causes bugs not caught by tests.
Again, this has happened to me personally, and has a high cost.
* Divergence between the branches means that all branches aren't
tested. Essentially nobody is testing b2g nightly, so it often
doesn't work. So why am I even landing patches on m-c?
In rare instances people have to help directly (thats what we should try to
reduce by
keeping trunk and v1.1 close as long v1.1 is still changing). Is this accurate
or do you
feel that uplifting is still a pain?
I don't think keeping v1.1 and trunk close is a goal of release
drivers. Their goal is to stabilize 1.1 by denying approval for risky
patches. That is directly at odds with keeping trunk and 1.1 close.
If we want to keep 1.1 and trunk close, there's a simple solution:
Make them the same branch. If that doesn't work, perhaps we need a
more creative solution. But certainly what we're doing now isn't
working towards this goal at all.
On Wed, Apr 3, 2013 at 3:59 PM, Andreas Gal <[email protected]> wrote:
The vendor predicts that they will file around 300 regressions during the QA cycle
(between now and August). I don't think we want to fix that on Aurora or Beta, and
"lets just slip 6 weeks" is unfortunately not how the hardware business works.
We are currently closely tied to hardware schedules because we are supporting individual
hardware launches with software. The goal is over time to switch to a reference software
model where we make a golden reference for a specific platform (a real hardware platform,
or even just a reference platform), and OEMs make products out of that (similar to the
Android model). Its clearly not a model where we have arrived at yet. For a while we will
have to continue to be closely involved in the actual product engineering. In a year or
two ideally we will be closer to the reference platform model, and in that world riding
trains makes sense, and slipping will be an option.
Andreas
On Apr 3, 2013, at 9:41 PM, Justin Lebar <[email protected]> wrote:
This is similar to what we did with 1.0. We rode the trains until
beta, then we branched.
The problem was, we branched too early. We thought the branch would
stabilize for a few weeks before we shipped, but in fact b2g18 has
been stabilizing for months and still isn't ready to ship as v1.0.1.
To avoid making the same mistake again, I think we'd probably want to
say that we're not going to branch. That essentially means we need to
be done when Aurora branches to Beta, because changes on Beta are
(rightly) highly restricted.
If we're not ready by the time we hit Beta, we slip by 6 weeks. We
don't put our beta users at risk for breakage to make our b2g
deadlines.
Note that we'd still be double-landing most patches on aurora and m-c,
because we will inevitably be under a huge amount of schedule
pressure. But that's a heck of a lot better than double-landing on
b2g18 and m-c.
On Wed, Apr 3, 2013 at 3:16 PM, Jonas Sicking <[email protected]> wrote:
Hi All,
It's come up a number of times that there are a fairly large number of
issues with the way that we're using "v1.1 branches", i.e. b2g18 for
Gecko and v1-train for gaia.
I won't go in to the various issues here since I think it's generally
agreed that it would be good if we could avoid them.
The current release plan for v1.1 means that we have code freeze on
August 12th. This is the date when, as I understand it, we absolutely
have to be done with the last line of code and it's pencils down for
everyone.
This actually means that we would have the ability to go back to
mozilla-central and gaia master right now and ride the normal release
trains for the Gecko 23 release, while still reaching the release
milestone for gecko before v1.1 is done.
This has a lot of attractive features:
* No backporting work for now!
* Much simpler backporting work once we branch for aurora on may 13th.
* We spend more time writing code and less time porting it.
* Less risk involved when backporting patches.
* We pick up a whole host of bug fixes and new features that's
happened since Gecko 18 branched. Including performance work.
However there are quite a few things that we need to check before we
can make such a move:
* Will it affect our partners ability to upgrade 1.0 users to the 1.1 release?
* Are there any big and scary landings that are planned for the Gecko
23 release that we'd rather not take. Gfx layers-refactoring comes to
mind.
* Will we be able to take security fixes that are landing in Gecko 23
all up until August 6th?
* Is mozilla-central and gaia master working well enough right now for
Firefox OS that this is an option?
* Will all relevant partner testing still be possible even through
we'll be on mozilla-central for some of it.
We also have to develop a plan for what to do if we need to fix some
Gecko issue that requires large enough changes that we can't land it
in the Firefox release trains. At that point we have to branch away
from the normal Firefox release branch. This means that all fixes that
go into the Firefox branch will have to also be ported to the B2G
branch.
All in all I think the advantages outweigh the disadvantages here.
Things from Firefox development landing in aurora and beta tend to be
very safe. Definitely safer than the sum of the large amounts of work
we'll still be doing for B2G. So if we can work through the issue list
above, I think we should do it.
Would love input from both developers and product on this.
Unfortunately I'll be on vacation thursday and friday so it would be
great to get help driving looking into the checklist above during that
time.
/ Jonas
_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g
_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g
_______________________________________________
dev-b2g mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-b2g
