>> If the past months have taught me anything, it's that "let's just >> branch and double-land everything" is unfortunately not a path to >> shipping quality software on time with happy, productive engineers. >> So this is not a one-sided trade-off. >> >> I get that there are partner constraints here, and perhaps riding the >> trains is incompatible with them, but I think there's a certain amount >> of creativity called for here, given that what we've been doing hasn't >> been working. I'd be curious to know if you have any ideas in this >> respect beyond "wait a few years". > > I don't think there are any silver bullets I can offer. What exactly is the > problem we are trying to solve here though. How often do people still have to > double-land themselves? The feedback I was getting is that the vast majority > of landings is done by our awesome uplifting mini-team.
Even with our wonderful sheriff team, the many branches we have are a huge pain. There are a lot of reasons. Here are some. * Constant fighting about "risk". Essentially all important patches have to go through two layers of approval (review, blocking+). Non-blocking patches have to go through three layers of approval (review, tracking+, approval+) these days. In most cases, the risk assessment is completely pro-forma. Engineers get what they want, in the end. But the process takes up time and energy. I contend we would /never/ put up with this sort of process BS in normal Firefox development. * Divergence between the branches makes landing hard. It's not at all uncommon for my patches to work on trunk and cause test failures on b2g18. And this problem will only get worse the longer we keep these branches around. * Divergence between the branches causes bugs not caught by tests. Again, this has happened to me personally, and has a high cost. * Divergence between the branches means that all branches aren't tested. Essentially nobody is testing b2g nightly, so it often doesn't work. So why am I even landing patches on m-c? > In rare instances people have to help directly (thats what we should try to > reduce by > keeping trunk and v1.1 close as long v1.1 is still changing). Is this > accurate or do you > feel that uplifting is still a pain? I don't think keeping v1.1 and trunk close is a goal of release drivers. Their goal is to stabilize 1.1 by denying approval for risky patches. That is directly at odds with keeping trunk and 1.1 close. If we want to keep 1.1 and trunk close, there's a simple solution: Make them the same branch. If that doesn't work, perhaps we need a more creative solution. But certainly what we're doing now isn't working towards this goal at all. >> On Wed, Apr 3, 2013 at 3:59 PM, Andreas Gal <[email protected]> wrote: >>> >>> The vendor predicts that they will file around 300 regressions during the >>> QA cycle (between now and August). I don't think we want to fix that on >>> Aurora or Beta, and "lets just slip 6 weeks" is unfortunately not how the >>> hardware business works. We are currently closely tied to hardware >>> schedules because we are supporting individual hardware launches with >>> software. The goal is over time to switch to a reference software model >>> where we make a golden reference for a specific platform (a real hardware >>> platform, or even just a reference platform), and OEMs make products out of >>> that (similar to the Android model). Its clearly not a model where we have >>> arrived at yet. For a while we will have to continue to be closely involved >>> in the actual product engineering. In a year or two ideally we will be >>> closer to the reference platform model, and in that world riding trains >>> makes sense, and slipping will be an option. >>> >>> Andreas >>> >>> On Apr 3, 2013, at 9:41 PM, Justin Lebar <[email protected]> wrote: >>> >>>> This is similar to what we did with 1.0. We rode the trains until >>>> beta, then we branched. >>>> >>>> The problem was, we branched too early. We thought the branch would >>>> stabilize for a few weeks before we shipped, but in fact b2g18 has >>>> been stabilizing for months and still isn't ready to ship as v1.0.1. >>>> >>>> To avoid making the same mistake again, I think we'd probably want to >>>> say that we're not going to branch. That essentially means we need to >>>> be done when Aurora branches to Beta, because changes on Beta are >>>> (rightly) highly restricted. >>>> >>>> If we're not ready by the time we hit Beta, we slip by 6 weeks. We >>>> don't put our beta users at risk for breakage to make our b2g >>>> deadlines. >>>> >>>> Note that we'd still be double-landing most patches on aurora and m-c, >>>> because we will inevitably be under a huge amount of schedule >>>> pressure. But that's a heck of a lot better than double-landing on >>>> b2g18 and m-c. >>>> >>>> On Wed, Apr 3, 2013 at 3:16 PM, Jonas Sicking <[email protected]> wrote: >>>>> Hi All, >>>>> >>>>> It's come up a number of times that there are a fairly large number of >>>>> issues with the way that we're using "v1.1 branches", i.e. b2g18 for >>>>> Gecko and v1-train for gaia. >>>>> >>>>> I won't go in to the various issues here since I think it's generally >>>>> agreed that it would be good if we could avoid them. >>>>> >>>>> The current release plan for v1.1 means that we have code freeze on >>>>> August 12th. This is the date when, as I understand it, we absolutely >>>>> have to be done with the last line of code and it's pencils down for >>>>> everyone. >>>>> >>>>> This actually means that we would have the ability to go back to >>>>> mozilla-central and gaia master right now and ride the normal release >>>>> trains for the Gecko 23 release, while still reaching the release >>>>> milestone for gecko before v1.1 is done. >>>>> >>>>> This has a lot of attractive features: >>>>> * No backporting work for now! >>>>> * Much simpler backporting work once we branch for aurora on may 13th. >>>>> * We spend more time writing code and less time porting it. >>>>> * Less risk involved when backporting patches. >>>>> * We pick up a whole host of bug fixes and new features that's >>>>> happened since Gecko 18 branched. Including performance work. >>>>> >>>>> However there are quite a few things that we need to check before we >>>>> can make such a move: >>>>> * Will it affect our partners ability to upgrade 1.0 users to the 1.1 >>>>> release? >>>>> * Are there any big and scary landings that are planned for the Gecko >>>>> 23 release that we'd rather not take. Gfx layers-refactoring comes to >>>>> mind. >>>>> * Will we be able to take security fixes that are landing in Gecko 23 >>>>> all up until August 6th? >>>>> * Is mozilla-central and gaia master working well enough right now for >>>>> Firefox OS that this is an option? >>>>> * Will all relevant partner testing still be possible even through >>>>> we'll be on mozilla-central for some of it. >>>>> >>>>> We also have to develop a plan for what to do if we need to fix some >>>>> Gecko issue that requires large enough changes that we can't land it >>>>> in the Firefox release trains. At that point we have to branch away >>>>> from the normal Firefox release branch. This means that all fixes that >>>>> go into the Firefox branch will have to also be ported to the B2G >>>>> branch. >>>>> >>>>> All in all I think the advantages outweigh the disadvantages here. >>>>> Things from Firefox development landing in aurora and beta tend to be >>>>> very safe. Definitely safer than the sum of the large amounts of work >>>>> we'll still be doing for B2G. So if we can work through the issue list >>>>> above, I think we should do it. >>>>> >>>>> Would love input from both developers and product on this. >>>>> Unfortunately I'll be on vacation thursday and friday so it would be >>>>> great to get help driving looking into the checklist above during that >>>>> time. >>>>> >>>>> / Jonas >>>>> _______________________________________________ >>>>> dev-b2g mailing list >>>>> [email protected] >>>>> https://lists.mozilla.org/listinfo/dev-b2g >>>> _______________________________________________ >>>> dev-b2g mailing list >>>> [email protected] >>>> https://lists.mozilla.org/listinfo/dev-b2g >>> > _______________________________________________ dev-b2g mailing list [email protected] https://lists.mozilla.org/listinfo/dev-b2g
