Great initiative, IMO. I favour going directly to openstack-, rather than stackforge-, for the migration reason that you mention.
Original Message From: Thomas Goirand Sent: Wednesday, 27 May 2015 09:17 To: OpenStack Development Mailing List (not for usage questions) Reply To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [packaging] Adding packaging as an OpenStack project -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi all, tl;dr: - - We'd like to push distribution packaging of OpenStack on upstream gerrit with reviews. - - The intention is to better share the workload, and improve the overall QA for packaging *and* upstream. - - The goal is *not* to publish packages upstream - - There's an ongoing discussion about using stackforge or openstack. This isn't, IMO, that important, what's important is to get started. - - There's an ongoing discussion about using a distribution specific namespace, my own opinion here is that using /openstack-pkg-{deb,rpm} or /stackforge-pkg-{deb,rpm} would be the most convenient because of a number of technical reasons like the amount of Git repository. - - Finally, let's not discuss for too long and let's do it!!! :) Longer version: Before I start: some stuff below is just my own opinion, others are just given facts. I'm sure the reader is smart enough to guess which is what, and we welcome anyone involved in the project to voice an opinion if he/she differs. During the Vancouver summit, operation, Canonical, Fedora and Debian people gathered and collectively expressed the will to maintain packaging artifacts within upstream OpenStack Gerrit infrastructure. We haven't decided all the details of the implementation, but spent the Friday morning together with members of the infra team (hi Paul!) trying to figure out what and how. A number of topics have been raised, which needs to be shared. First, we've been told that such a topic deserved a message to the dev list, in order to let groups who were not present at the summit. Yes, there was a consensus among distributions that this should happen, but still, it's always nice to let everyone know. So here it is. Suse people (and other distributions), you're welcome to join the effort. - - Why doing this ================ It's been clear to both Canonical/Ubuntu teams, and Debian (ie: myself) that we'd be a way more effective if we worked better together, on a collaborative fashion using a review process like on upstream Gerrit. But also, we'd like to welcome anyone, and especially the operation folks, to contribute and give feedback. Using Gerrit is the obvious way to give everyone a say on what we're implementing. As OpenStack is welcoming every day more and more projects, it's making even more sense to spread the workload. This is becoming easier for Ubuntu guys as Launchpad now understand not only BZR, but also Git. We'd start by merging all of our packages that aren't core packages (like all the non-OpenStack maintained dependencies, then the Oslo libs, then the clients). Then we'll see how we can try merging core packages. Another reason is that we believe working with the infra of OpenStack upstream will improve the overall quality of the packages. We want to be able to run a set of tests at build time, which we already do on each distribution, but now we want this on every proposed patch. Later on, when we have everything implemented and working, we may explore doing a package based CI on every upstream patch (though, we're far from doing this, so let's not discuss this right now please, this is a very long term goal only, and we will have a huge improvement already *before* this is implemented). - - What it will *not* be ======================= We do not have the intention (yet?) to publish the resulting packages built on upstream infra. Yes, we will share the same Git repositories, and yes, the infra will need to keep a copy of all builds (for example, because core packages will need oslo.db to build and run unit tests). But we will still upload on each distributions on separate repositories. So published packages by the infra isn't currently discussed. We could get to this topic once everything is implemented, which may be nice (because we'd have packages following trunk), though please, refrain to engage in this topic right now: having the implementation done is more important for the moment. Let's try to stay on tracks and be constructive. - - Let's keep efficiency in mind =============================== Over the last few years, I've been able to maintain all of OpenStack in Debian with little to no external contribution. Let's hope that the Gerrit workflow will not slow down too much the packaging work, even if there's an unavoidable overhead. Hopefully, we can implement some liberal ACL policies for the core reviewers so that the Gerrit workflow don't slow down anyone too much. For example we may be able to create new repositories very fast, and it may be possible to self-approve some of the most trivial patches (for things like typo in a package description, adding new debconf translations, and such obvious fixes, we shouldn't waste our time). There's a middle ground between the current system (ie: only write access ACLs for git.debian.org with no other check what so ever) and a too restrictive fully protected gerrit workflow that may slow down everyone too much. Currently, there's a small amount of people involved in the packaging. While we can expect a lot of operators will be interested to work on core packages like Nova, Neutron and so on, I am at the same time expecting that nobody will care about a given indirect python module dependency (and I may still continue to be the only one working on these...). We really don't want to be stuck in situations with nobody reviewing these. - - /stackforge or /openstack namespace ===================================== There's been this question floating around. Should we try joining directly as part of OpenStack, as many suggested, or should we go through a first round (during one dev cycle, until Liberty is released?) through stackforge. I have to admit that I'm not strongly opinionated about this myself. Sure, being an official OpenStack big-tent project would be nice, and it would feel worm, but also there's the issue that we don't have any commits within the project yet, and therefore finding who's core and who's the PTL could be an issue if we want to follow the official guidelines. However, without even discussing the topic between us, it's obvious that people like James Page (plus others from his team) and myself would be core approvers for the Debian packaging. Fedora guys can decide for themselves (I must write names I have in mind for the RPM based-OS side of things: M. Runge, Haikel Gemmar and Alan Pevec, for example. Please add the missing ones here...). Another thing is that going for stackforge now means that one day, we'll have to suffer from a migration to the openstack namespace, which may be 1/ a lot of work and 2/ disruptive. Avoiding such a migration would be good. All together, the packaging team will happily accept whatever is decided by the TC. What counts for us is that a decision is taken quickly, so that we can start implementing it and share our repositories. So, dear TC, please let us know ASAP after your next meeting. Anyhow, please feel free to discuss the issue within the governance patch which I started: https://review.openstack.org/#/c/185187/ - - Specific packaging namespace ============================== Since distribution packaging through Git implies running a lot of git repository (one per package), it's been discussed that we may want to use a specific namespace for distributions. For example, /stackforge-packaging or /openstack-packaging. Let's keep in mind that, for OpenStack packaging by the distributions, we're talking about more than 200 packages. 235 so far on the OpenStack packaging group in Debian [1], and it's increasing all the time... Personally, I believe it'd be nice to have a specifc namespace for distributions, for multiple reasons. The first one is that we already have a lot of projects inside /stackforge or /openstack, and that searching through them isn't easy. Now, consider that we'll have duplicates for each and every project. Also, we don't have a complete match between OpenStack package names. For example, in Debian/Ubuntu, we have "python-" as prefix for each and every python libraries (including for example all oslo packages, but not only). Plus we'd be forced to use a prefix, for example "pkg-nova", to avoid collision. However, it'd be nice to keep the same repository names as per the name of the source package in the distributions, otherwise we'd have to fix our tooling which obviously will become slightly more complex. - - Distribution specific git repositories or specific branches ============================================================= There's 2 issues if we keep the same repository names for multiple distribution. First, we'd like to have a different set of core reviewers for a given distribution family. This implies that, if we use the same repository for RPM and DEB, then we'd have to set ACLs on per-branch basis, which may be harder to maintain (compared to giving access to the full set of Git repositories for a given distro family). Yes, we could give access to both for core reviewers, and trust them to behave. This trust model has been proved as successful within the Debian community, but still, adding and removing ACLs for new core reviewers would be harder to maintain. Then, as written above, it is desirable to have a matching pair between distribution source package names and the name of the underlying git repository (so that for building X, you git clone X, not prefix-X). However, distributions don't have the same names for packages. For example, RPM supports upper case, while dpkg systems don't. Also, Red Hat uses a prefix "openstack-" for all core projects (like "openstack-nova"), while Debian based distribution don't, and so on... - - Finally ========= Finally, well, please don't discuss the color of the bike-shed for too long. We don't really care the color as long as we have a shed to share the hosting of our packages Git. Yes, I'm open to discussing all of the issues above, but if and only if it doesn't become a blocker for implementing packaging on upstream Gerrit. For the /openstack namespace, can we agree to set a deadline in 6 days, before the next TC meeting? We really need to get started soon, at the beginning of the liberty cycle, so that we can achieve important tasks like merging our source packages between Debian and Ubuntu (for example) and still be on-time for the release of Liberty. The task will be non-trivial, if we want to accommodate all parties, and retain the features our users had enjoyed over the past few years. Last word: thanks to everyone on the infra team (and others) which were supportive of the project. All of this is very exciting, and it's really great that this is finally happening. Cheers, Thomas Goirand (zigo) [1] https://qa.debian.org/developer.php?login=openstack-de...@lists.alioth.debian.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCAAGBQJVZXzYAAoJENQWrRWsa0P+M8gP/21TcEfBs/PTxUFRDMSfP97a JCVj5JIy6GCFjDC+8mbX9WGWFEbU0hRn2CG/dRJNbcrkjeQJaDosSd3BMmEAsrQO ONyu0Qbm2YgclZ9OyUaewE8NPSH8cGRjxS+IdOm5kzhNqX538ohsxD2XNMknx5rU cyq3TTDKWL3ot8+SVdUqfvzmZZdESF+8lucneqOiRk9wU/UVt6fdsovuv3QSqM2h k7etO4Oev46nQFCVvSmofv2mfGTWn5iWDbcCj7qf/blI5uiL4GkTkzFbkr/+f2De 7C4p9rO8WsEaK+Ptg2hayiOm36i6JPnvO6YSOjm6yTeY4LQ3UkULN2/THvftSkYX I+oICj7XbJMhaUDbg9mfJhVZ+O5L0v9YHV2v12NIWMbDGb6EzSYsLqmvANwYN5wM Y8zihyETZFYsJVmPdqHN6ZPGeuKAroFo0F5rldMJ/++J5exApf6DJyd+JrE7vmyV F3sksS+BhmPgmEBbSvcA0ZP8gqEfPWMK5zT1rWBakKy1OHtDaLwnzsCAkaS6kXUJ G45Sr965H6pkO10EMpIbjFeCcVIP7UpmHDxq1PQbhUx4aogjVGA1mIWwuD0Cscaj hD18bJyZxe0t0MzhkijxWnQ6holJ1C36SvSD/jha6oZ7k9FHIAwo1MklTJcCEo7J PBnrpJTi9IkhDmfp21/s =5T0x -----END PGP SIGNATURE----- __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev