>> I contend we would /never/ put up with this sort of process >> BS in normal Firefox development. > > This is so apples and oranges it's astonishing. How can you compare the > landing requirements of desktop releases (which involve no partners, have > massive pre-release testing populations, is a mature product, and has no hard > feature scope) to B2G releases?
Forgot to mention a lack of reliable, automated testing. -Alex On Apr 3, 2013, at 1:42 PM, Alex Keybl <[email protected]> wrote: > Justin, let's take a step back and not make invalid assertions. > >> * Constant fighting about "risk". Essentially all important patches >> have to go through two layers of approval (review, blocking+). >> Non-blocking patches have to go through three layers of approval >> (review, tracking+, approval+) these days. > > This is false. Any bug can request approval without being tracking+ (same as > desktop). Tracking nomination are a good way to understand whether the user > impact is enough to warrant uplift, or to get automatic uplift in the next > release. > >> In most cases, the risk assessment is completely pro-forma. Engineers >> get what they want, in the end. But the process takes up time and >> energy. > > If it's not worth the 2 minutes for an engineer to request approval, it's > likely not worth uplifting the code change. And if approved, the bug is > uplifted automatically by sheriffs. > > You're right though - you all could lie to us about risk or reward. We're > trusting you aren't. > >> I contend we would /never/ put up with this sort of process >> BS in normal Firefox development. > > This is so apples and oranges it's astonishing. How can you compare the > landing requirements of desktop releases (which involve no partners, have > massive pre-release testing populations, is a mature product, and has no hard > feature scope) to B2G releases? > > -Alex > > On Apr 3, 2013, at 1:27 PM, Justin Lebar <[email protected]> wrote: > >>>> If the past months have taught me anything, it's that "let's just >>>> branch and double-land everything" is unfortunately not a path to >>>> shipping quality software on time with happy, productive engineers. >>>> So this is not a one-sided trade-off. >>>> >>>> I get that there are partner constraints here, and perhaps riding the >>>> trains is incompatible with them, but I think there's a certain amount >>>> of creativity called for here, given that what we've been doing hasn't >>>> been working. I'd be curious to know if you have any ideas in this >>>> respect beyond "wait a few years". >>> >>> I don't think there are any silver bullets I can offer. What exactly is the >>> problem we are trying to solve here though. How often do people still have >>> to double-land themselves? The feedback I was getting is that the vast >>> majority of landings is done by our awesome uplifting mini-team. >> >> Even with our wonderful sheriff team, the many branches we have are a >> huge pain. There are a lot of reasons. Here are some. >> >> * Constant fighting about "risk". Essentially all important patches >> have to go through two layers of approval (review, blocking+). >> Non-blocking patches have to go through three layers of approval >> (review, tracking+, approval+) these days. >> >> In most cases, the risk assessment is completely pro-forma. Engineers >> get what they want, in the end. But the process takes up time and >> energy. I contend we would /never/ put up with this sort of process >> BS in normal Firefox development. >> >> * Divergence between the branches makes landing hard. It's not at all >> uncommon for my patches to work on trunk and cause test failures on >> b2g18. And this problem will only get worse the longer we keep these >> branches around. >> >> * Divergence between the branches causes bugs not caught by tests. >> Again, this has happened to me personally, and has a high cost. >> >> * Divergence between the branches means that all branches aren't >> tested. Essentially nobody is testing b2g nightly, so it often >> doesn't work. So why am I even landing patches on m-c? >> >>> In rare instances people have to help directly (thats what we should try to >>> reduce by >>> keeping trunk and v1.1 close as long v1.1 is still changing). Is this >>> accurate or do you >>> feel that uplifting is still a pain? >> >> I don't think keeping v1.1 and trunk close is a goal of release >> drivers. Their goal is to stabilize 1.1 by denying approval for risky >> patches. That is directly at odds with keeping trunk and 1.1 close. >> >> If we want to keep 1.1 and trunk close, there's a simple solution: >> Make them the same branch. If that doesn't work, perhaps we need a >> more creative solution. But certainly what we're doing now isn't >> working towards this goal at all. >> >>>> On Wed, Apr 3, 2013 at 3:59 PM, Andreas Gal <[email protected]> wrote: >>>>> >>>>> The vendor predicts that they will file around 300 regressions during the >>>>> QA cycle (between now and August). I don't think we want to fix that on >>>>> Aurora or Beta, and "lets just slip 6 weeks" is unfortunately not how the >>>>> hardware business works. We are currently closely tied to hardware >>>>> schedules because we are supporting individual hardware launches with >>>>> software. The goal is over time to switch to a reference software model >>>>> where we make a golden reference for a specific platform (a real hardware >>>>> platform, or even just a reference platform), and OEMs make products out >>>>> of that (similar to the Android model). Its clearly not a model where we >>>>> have arrived at yet. For a while we will have to continue to be closely >>>>> involved in the actual product engineering. In a year or two ideally we >>>>> will be closer to the reference platform model, and in that world riding >>>>> trains makes sense, and slipping will be an option. >>>>> >>>>> Andreas >>>>> >>>>> On Apr 3, 2013, at 9:41 PM, Justin Lebar <[email protected]> wrote: >>>>> >>>>>> This is similar to what we did with 1.0. We rode the trains until >>>>>> beta, then we branched. >>>>>> >>>>>> The problem was, we branched too early. We thought the branch would >>>>>> stabilize for a few weeks before we shipped, but in fact b2g18 has >>>>>> been stabilizing for months and still isn't ready to ship as v1.0.1. >>>>>> >>>>>> To avoid making the same mistake again, I think we'd probably want to >>>>>> say that we're not going to branch. That essentially means we need to >>>>>> be done when Aurora branches to Beta, because changes on Beta are >>>>>> (rightly) highly restricted. >>>>>> >>>>>> If we're not ready by the time we hit Beta, we slip by 6 weeks. We >>>>>> don't put our beta users at risk for breakage to make our b2g >>>>>> deadlines. >>>>>> >>>>>> Note that we'd still be double-landing most patches on aurora and m-c, >>>>>> because we will inevitably be under a huge amount of schedule >>>>>> pressure. But that's a heck of a lot better than double-landing on >>>>>> b2g18 and m-c. >>>>>> >>>>>> On Wed, Apr 3, 2013 at 3:16 PM, Jonas Sicking <[email protected]> wrote: >>>>>>> Hi All, >>>>>>> >>>>>>> It's come up a number of times that there are a fairly large number of >>>>>>> issues with the way that we're using "v1.1 branches", i.e. b2g18 for >>>>>>> Gecko and v1-train for gaia. >>>>>>> >>>>>>> I won't go in to the various issues here since I think it's generally >>>>>>> agreed that it would be good if we could avoid them. >>>>>>> >>>>>>> The current release plan for v1.1 means that we have code freeze on >>>>>>> August 12th. This is the date when, as I understand it, we absolutely >>>>>>> have to be done with the last line of code and it's pencils down for >>>>>>> everyone. >>>>>>> >>>>>>> This actually means that we would have the ability to go back to >>>>>>> mozilla-central and gaia master right now and ride the normal release >>>>>>> trains for the Gecko 23 release, while still reaching the release >>>>>>> milestone for gecko before v1.1 is done. >>>>>>> >>>>>>> This has a lot of attractive features: >>>>>>> * No backporting work for now! >>>>>>> * Much simpler backporting work once we branch for aurora on may 13th. >>>>>>> * We spend more time writing code and less time porting it. >>>>>>> * Less risk involved when backporting patches. >>>>>>> * We pick up a whole host of bug fixes and new features that's >>>>>>> happened since Gecko 18 branched. Including performance work. >>>>>>> >>>>>>> However there are quite a few things that we need to check before we >>>>>>> can make such a move: >>>>>>> * Will it affect our partners ability to upgrade 1.0 users to the 1.1 >>>>>>> release? >>>>>>> * Are there any big and scary landings that are planned for the Gecko >>>>>>> 23 release that we'd rather not take. Gfx layers-refactoring comes to >>>>>>> mind. >>>>>>> * Will we be able to take security fixes that are landing in Gecko 23 >>>>>>> all up until August 6th? >>>>>>> * Is mozilla-central and gaia master working well enough right now for >>>>>>> Firefox OS that this is an option? >>>>>>> * Will all relevant partner testing still be possible even through >>>>>>> we'll be on mozilla-central for some of it. >>>>>>> >>>>>>> We also have to develop a plan for what to do if we need to fix some >>>>>>> Gecko issue that requires large enough changes that we can't land it >>>>>>> in the Firefox release trains. At that point we have to branch away >>>>>>> from the normal Firefox release branch. This means that all fixes that >>>>>>> go into the Firefox branch will have to also be ported to the B2G >>>>>>> branch. >>>>>>> >>>>>>> All in all I think the advantages outweigh the disadvantages here. >>>>>>> Things from Firefox development landing in aurora and beta tend to be >>>>>>> very safe. Definitely safer than the sum of the large amounts of work >>>>>>> we'll still be doing for B2G. So if we can work through the issue list >>>>>>> above, I think we should do it. >>>>>>> >>>>>>> Would love input from both developers and product on this. >>>>>>> Unfortunately I'll be on vacation thursday and friday so it would be >>>>>>> great to get help driving looking into the checklist above during that >>>>>>> time. >>>>>>> >>>>>>> / Jonas >>>>>>> _______________________________________________ >>>>>>> dev-b2g mailing list >>>>>>> [email protected] >>>>>>> https://lists.mozilla.org/listinfo/dev-b2g >>>>>> _______________________________________________ >>>>>> dev-b2g mailing list >>>>>> [email protected] >>>>>> https://lists.mozilla.org/listinfo/dev-b2g >>>>> >>> >> _______________________________________________ >> dev-b2g mailing list >> [email protected] >> https://lists.mozilla.org/listinfo/dev-b2g > _______________________________________________ dev-b2g mailing list [email protected] https://lists.mozilla.org/listinfo/dev-b2g
