On Fri, Aug 7, 2009 at 1:56 PM, Julian Edwards<julian.edwa...@canonical.com> wrote: > On Friday 07 August 2009 17:28:19 Jonathan Lange wrote: >> On Fri, Aug 7, 2009 at 5:25 PM, Jonathan Lange<j...@canonical.com> wrote: >> > Hello everyone, >> > >> > Another spec from me today, this one based on some notes that cprov, >> > mwhudson, james_w and I made at UDS in Barcelona. >> > >> > This spec shows one way for Launchpad to build branches (esp source >> > package branches) to archives, including PPAs and primary archives. >> >> *ahem* >> >> https://dev.launchpad.net/BuildBranchToArchive > > Thanks for starting that page off Jono.
Yes, thanks for leading this change. It will be an amazing feature when it's out. > Rather than writing a new "dispatcher" I think it might be a good idea to > amend our existing buildd-manager to do these branch jobs, so that we can use > the existing build farm. I'm vastly simplifying here, but all we need to do > is make a "branch job" chroot for that type of job. I think installing `bzr-buildddeb` on the existing chroots is better, mainly because it avoids maintaining 2 sets of chroots for all supported distroarchseries. The only inconvenient is performance, but until we identify it as a major problem, I would simply go with the existing chroots and install the needed stuff as part of the build process. It will be fine, as long as `bzr-builddeb` is still functional in all the supported suites (even if we have to include a parallel PPA providing it). 1. Download the appropriate chroot 2. Unpack and set it up 3. Override sources_list 4. Upgrade 5. Install `bzr-builddeb` 6. Run it! ... N. Clean up Steps 1 - 4 and N already exist and can be reused from the debian slave. Step 5 is trivial and step 6 should be something like `bzr get -r REV BRANCH_URL job` then `bzr bd job -- -S -uc -us` obs: James, I couldn't make bd build a remote branch directly in jaunty, I had to fetch the branch first then build it. Am I doing something wrong ? > We currently have unsigned uploads when the buildd-manager uploads binaries > and this could be extended to sources coming from the branch jobs. The only > issue I can think of is the lack of a dsc file signature. Even if we did sign > it with an LP signature, I'm not sure how meaningful that would be since we > trust our build farm anyway. It seems prudent to audit our code to see how > much of it makes assumptions around dsc signing (I can think of a few). Only one template assumes signed source uploads for PPAs, IIRC, all other SPR.dscsigningkey callsites respect the schema (this column is nullable). The soyuz upload processor is already prepared accept unsigned source uploads, we may need to encapsulate this as a different upload policy, but this is will be a tiny change. I'm little concerned about the schema changes we have to make. I'm proposing them based on the fact that we agreed on using the existing build-farm to assembly source packages from branches. It implies in: 1. Perform branch-jobs using the existing `launchpad-buildd`s. Currently the only registered slave is the 'debian' one (DebianBuildManager) we can easily create a new one, let's say, BzrBuildManager reusing the chroot setup/cleanup and adding the bzr-builddeb steps replacing sbuild. The instead of calling build(..., 'debian', ...) over XMLRPC, we would pass 'bzr' instead. 2. Dispatch and collect branch-jobs using `buildd-manager` One way of implementing this would be extending BuildQueue to support both, source and branch jobs. The existing names are confusing, but in the same way the Build content class represents which source has to be build, the proposed BranchJob represents which branch has to be build. BuildQueue represents a queue of jobs that have to be dispatched to builders (better named 'BuilderQueue', anyway ...) If we add a BranchJob FK on BuildQueue we can continue to use the same dispatching mechanisms against the same pool of builders and also manage priority in the same way for both types of build. This way dispatching and collecting results can be delegated from BQ to one of its children, Build or BranchJob, and implemented according each context. What do you guys think about this ? [] -- Celso Providelo <celso.provid...@canonical.com> IRC: cprov, Jabber: cp...@jabber.org, Skype: cprovidelo 1024D/681B6469 C858 2652 1A6E F6A6 037B B3F7 9FF2 583E 681B 6469 _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
