hi, On Do, 2013-11-28 at 17:01 +0000, Dave Morley wrote: > On 28/11/13 12:13, Oliver Grawert wrote: > > Hi, > > > > As some might have noticed we recently had a few bad image releases into > > the Trusty channel that contained regressions. > > We also tested out some changes to the build policies of the proposed > > images that I would like to establish further. > > > > Within the last weeks we raised the amount of built images per day to a > > higher frequency (3 at most currently) ... the reason for this: > > currently finding a regression in an image when only building a single > > image per day results in a pretty big change set. Finding the offending > > package that introduced a regression can take hours of weeding through > > package changelogs and is often based on guesswork ... usually we are > > guessing pretty well, but this still burns a lot of developer time we > > should rather spend on fixing bugs and writing tests or features. > > > > with building at least 3 images per day the change sets got a lot > > smaller, finding issues becomes a charm and offending packages are easy > > to identify ... > > > > My proposal here is to re-enable cron driven builds again. > > For a start this has to be every 8h since our testing infrastructure can > > not cope with more frequent builds and over time (as our testing > > infrastructure improves) to get to a frequency of doing a build every > > 4h, this should give us a small enough set of changes to be a lot > > quicker in finding regressions. I would like to switch cron back on by > > end of this week on the above mentioned 8h schedule, it seems all > > involved teams agreed that this is a good plan, please speak up if this > > plan does not suit the way your team works (or speak up in support :) ) > > Yay!!!! What times? The likelihood is that each time zone will only > ever see 2 releases a day I'm assuming. But it would also be good to > know if you should wait another 5 minutes for the next release before > downloading, especially if I'm on 3g. teh times will indeed be publically announced here, on a wikipage and on G+ ;) you wont miss them ... > > > > > The second part of my proposal goes a bit further than just adding a > > cron entry ... > > > > Today the release of an image is handled by the Landing Team, which was > > traditionally only responsible for landing new upstream code inside the > > image ... this task alone already hogs most resourcesof the team. > > Releasing an image is usually pretty quickly decided as one topic of a > > daily meeting based on a quick glance over the automated tests and by > > asking for feedback from people that have done a short dogfooding > > smoketest ... > > > > I personally think the release process deserves a lot more attention, > > way more input from other teams and a wider dogfooder audience. The > > Landing team should concentrate on the landings of new code, the time > > they need to invest in additional image testing is time they can not > > spend on testing new landings which slows all of us down by creating an > > artificial bottleneck now that all landings need to go through that > > team. They should be able to fully concentrate on this process instead. > > > > What I like to throw in as a discussion point is to form a new > > "touch-release" team that consists of people from QA that bring in the > > buglist of collected bugs and triages and also checks errors.ubuntu.com > > regulary (we might need a "touch" filter there), one representative of > > the avengers team (who dogfood the stable images), someone from the > > ubuntu-cdimage team and a representative of the Landing team to give > > input and get feedback on the recent landings from actual endusers. > > > > This team should be a public focused team holding daily IRC meetings the > > whole community can participate in (as opposed to privately held > > team-only hangouts), I know there are plenty of users of the > > trusty-proposed/devel-proposed images and I think we should really try > > to have them participate in the release process and enable them to give > > us feedback to: > > > > a) regressions they run into that went unnoticed by the developers > > b) new bugs the QA/triage teams have not yet seen > > c) critical blockers we didn't even consider or see yet because they > > only happen after a day or two of constant usage. > > > > With the higher frequency of proposed builds it should then be possible > > to pick the image with the best automated results from a former day (or > > even two days so testers had more time for long term testing) that is > > considered good by all participating parties and their individual > > requirements. > > > > Lets put more focus onto the Image releases and take load of the Landing > > team to speed us all up and have the community more involved to actually > > increase the quality of our images and be truly regression free. > > > > I think that something based on these lines would make a certain amount > of sense. I would hold off on a meeting everyday though if the > community are involved it is hard for everyone to turn up everyday > particularly with timezone constraints and instead schedule them for > Tuesdays and Thursday. This would mean you roll out a stable-ish image > based on the meeting and leaves you with a nice stable release over the > weekend for people to really dogfood. > > It might also be good, as we start the process of reducing the test > failure rate, that we set a level that we won't release under. For > example if the overall image percentage drops below 90% it is simply > ignored till the next release. This cuts down drastically on the image > pool to release from. > > Just some addition thoughts. > > this makes a lot of sense ... I would still like to do meetings more often than once a week, not every community tester needs to participate in each meeting as long as we can aggregate the feedback somehow ... probably daily is a bit to much though
ciao
oli
signature.asc
Description: This is a digitally signed message part
-- Mailing list: https://launchpad.net/~ubuntu-phone Post to : ubuntu-phone@lists.launchpad.net Unsubscribe : https://launchpad.net/~ubuntu-phone More help : https://help.launchpad.net/ListHelp
