Monty++! Just FYI
GridDynamics are working well for Packaging RPMs. http://wiki.openstack.org/NovaInstall/RHEL6Notes-Diablo-3 Its also works on Scientific Linux6.1 (Personally, I prefer free OS :D ) Here is there repos. http://yum.griddynamics.net/yum/diablo-3/openstack/ They maintains it on the github. https://github.com/griddynamics/openstack-rhel I hope their great work would become official repos. Nati (NTT) 2011/8/24 Monty Taylor <mord...@inaugust.com>: > Hi all! > > First of all, I spoke about something yesterday in terms of being a > done-deal, when it should have been prefaced with "I'm looking at". My > bad, and I certainly apologize for freaking anyone out. > > That being said - there are several issues with the current packaging > setup that I've been tasked with tackling. This email will outline the > issues, and then I'll discuss what I've been thinking in terms of solutions. > > ***** > This is a proposal. We want to get feedback on this before the summit > but will not be changing the way we do packaging before the summit. > ***** > > For reference, the current setup is: > Jenkins creates source packages from a team > (~openstack-ubuntu-packagers) packaging branch for every trunk commit > for Ubuntu and uploads them to a PPA on Launchpad > > Let me start be enumerating the issues with the current setup that we're > trying to solve. (These are not in order of importance) > > - Lack of Debian support. One of the largest current deployments of > OpenStack is Rackspace, and they deploy on Debian. (disclaimer, they > also pay my salary) This means they are internally maintaining their own > set of packages, and not interacting with ours at all. That sort of sucks. > > - Lack of collaboration. Related to the above, but slightly different. > Since the Rackspace internal deployment team have to maintain their own > packages, it means they are not using the published packages and we are > not benefiting from their real-world deployment experience. > Additionally, the Cloudbuilders consulting team doesn't use our > published packages, and similarly we aren't really benefiting from them > directly working with us on packaging. > > - PPAs are Async. This makes integration testing harder. We're about to > roll out deployment/integration testing, which involves installing the > latest set of packages in environments and then testing that it works. > In that workflow, the queue/async nature of using LP as our package > builder becomes more problematic. I cannot trigger a direct test of a > set of packages as part of the testing of a commit if part of the > process is "upload into the LP queue and then wait for a while" - > especially since LP still has not added any sort of callback mechanism > to report completion. This means Jenkins is locally building packages > anyway so that it can pass them to deployment environments to test them. > If it's building them locally, and that's what's being tested, why > wouldn't that be what's published? > > - Divergent dev process from the rest of OpenStack. The PPB just > recently voted that all OpenStack projects must use the same set of > tooling. Now granted, packaging is not a core project, so we could get > legalistic and claim that we're not required to comply - but that seems > a little disingenuous - especially since I was a strong proponent of the > one-set-of-tooling model. Additionally, since we have a strong code > review/testing system that's integrated in to our build and deployment > system, it seems very odd not to take advantage of it. (having new > packages created triggered by both approval of packaging branch changes > AND new upstream tarballs seems kindof ideal, no?) > > - Not integrated with CI. PPA build errors are not reported inside of > our CI system where all of the rest of our errors are reported. > Additionally, those build errors are not reproducible by devs locally, > as they cannot spin up a launchpad PPA builder. > > - Lack of RPM support. Turns out RPM distros are still kind of popular, > and at least one of our major contributors (hi, NTT) rolls out on > RHEL/CentOS, and one of our partners who is going to be contributing > testing hardware (MSFT/Novell) is associated with a distro (SuSE) that > is RPM based. We're a few steps away from this, but it warrants mentioning. > > - Murky backported deps. Some of the backported deps in the PPAs came > from a dir on my laptop. Some are from Theirry's laptop. Some are from > Soren's laptop. There is neither documentation on how they got there, > how we decided to put them there, how we built them or how to continue > to collaborate on them. > > - Direct injection to Ubuntu not via Debian. The publically stated > best-practice for things entering Ubuntu is to be added to/uploaded to > Debian and then synced, unless there is some compelling reason not to do > this. In this case, we had Ubuntu devs and no Debian devs in the core, > so we keep improving packages and adding packages to Ubuntu but are not > doing the same to Debian. > > - PPA Fragmentation. The PPB voted that we are one project, yet we have > a separate PPA for each project. > > Ok. I lied - those might have been in a general order of importance. > Turns out though, that in terms of addressing them, they fall in to two > different groupings: > > 1) How we manage and work on the packaging code itself > 2) How we build and publish packages > > The two are related, but not linked. Of the two, the most absolutely > important is 2, so I'll address it first. I'm also going to be a bit > more blunt about it than the other one. > > == How we build and publish packages == > > This deals with: > - Lack of Debian support. > - PPAs are Async. > - Not integrated with CI. > - Lack of RPM support. > - PPA Fragmentation. > > I'm going to be rolling out http://packages.openstack.org, which will at > first contain packages for lucid, maverick, natty, oneiric, squeeze and > wheezy. There will be at the very least a trunk repo and a repo > per-release (so a diablo one) Will probably use the debian trick of > pointing trunk at diablo at first, and then creating an essex repo and > pointing essex at that - but I'm up for suggestions there. We have a > Jenkins, we have plenty of builder slaves and we have pbuilder for > chroot clean builds - there is no compelling reason for us to wait on > the Launchpad build queue. AND - I can hand a puppet module to a dev > that they can run on a cloud server and they can have access to an env > which is EXACTLY like the one that build errors that they experience > occured in. > > So big wins here are: > - single place to get OpenStack as a cohesive whole > - addition of pure Debian packages > - possibility of adding YUM repos to packages.openstack.org > - synchronous publishing for better testing > - end to end visibility of what build actions were taken > > Note that there is nothing stopping PPA packages from existing - we > could even have jenkins continue upload things to PPAs as well - it's > just not really a primary target. > > == How we manage and work on the packaging code itself == > > Now to the one that's a little more up in the air because it's the one > that's going to cause people to throw their hands up in the air and > scream and gnash their teeth. Sigh. Sorry. > > To address: > - Lack of collaboration. > - Divergent dev process from the rest of OpenStack. > - Murky backported deps. > - Direct injection to Ubuntu not via Debian. > > I'm suggesting: > - we move all packaging to git > - we manage it consistently via git-buildpackage > - we code review packaging branches in gerrit > - we publish packages only through reviewed branches > > Because of the last, point, I'd like to point out that this applies to > backported packages we need in our repo as well. Building any package > from the set of important OpenStack pacakges should go through the same > process. Managing the packaging for our packages and for backports will > be slightly different. > > What exactly does that mean? > > For our packages: > > As Thomas Goirand suggested, we will add a packaging branch to the repo > of each of our projects and have git-buildpackage do the right thing. We > can have gerrit set different ACLs for the packaging branch so that a > different set of folks (the packaging team) can handle the reviewing of > that branch. Combining this with a pristine-tar and having Jenkins push > some tags back to the repo on package release, it means that we can > trace any release of software back to our code repo, and that only the > repo is needed to re-create any particular release. > > The jenkins job for this is triggered off of both new commits to trunk > and to new commits to packaging. > > For backported packages: > > Having branches on top of upstream source is almost useless here, as in > most cases we're doing one-off backports. So here we simply make a new > repo via git-import-dsc, upload it and register it with jenkins. In the > common case, there won't be much more work to do - but in the ongoing > case, it means that anyone in our dev community knows where to go to get > at packaging of specific versions of things carried in our repo. > > The jenkins jobs for these are triggered off of new commits to packaging. > > For both: > > When Jenkins is triggered, it will pull the branches and build packages > for all available targets in base chroots using git-pbuilder/cowbuilder. > Each build will have its full log in Jenkins just as any other build. > >From a repo perspective, for each target, Jenkins will attempt to build > using packaging in the debian branch unless there is a branch named > debian/${target} - so if the packaging needs to be overridden for, say, > lucid (because it's older than dirt) It's easy enough to just create a > debian/lucid branch and leave debian to handle the other branches. > > These packages are then all published to packages.openstack.org and jobs > testing deployment are kicked. At a point in the future (if we start > doing pre-merge deployment testing) we'll need to make a pending archive > so that we can re-publish same versions due to failure -but we're not > there yet. > > As a bonus, git-buildpackage does some nice tagging of things, which we > can have Jenkins push back, so we can keep our repos up to date with > meta information. Since these are on ancillary branches, that pushing > doesn't have to interrupt normal dev work. Incidentally, since that meta > info is in the repo, we cease to need the tarball_versions and > ppa_versions files that we currently manage on the filesystem in jenkins. > > I know there are other ways to accomplish some of these things, but this > is the one I'm proposing. The fact that it fits in to our development > methodology is appealing, as is the possibility of getting pvo and jesse > and thomas looped in to working directly on this set of packages. > > Monty > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > -- Nachi Ueno email:nati.u...@gmail.com twitter:http://twitter.com/nati _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
