On Wed, Jul 08, 2020 at 10:52:38AM +0100, Stefan Hajnoczi wrote: > Dear QEMU community, > QEMU currently has a static website, wiki, git repo hosting, and > special-purpose cronjobs/containers running in VMs. There is currently > no system administrator looking after our infrastructure so the most > urgent tasks fall onto me, the remainder are ignored/postponed. The > current situation exposes qemu.org to the risk of downtime and > security issues.
I would further say that maintaining infrastructure is not a good use of our contributors' time. It is better to have our contributors spend time delivering features that are beneficial to our users, rather than playing at sysadmin. > Another limitation is that each piece of infrastructure is managed > separately and one-time contributors cannot easily propose changes > because they do not have access. It would be much better to use our > existing code review process so that anyone can make changes to > infrastructure by sending a patch. When I was thinking about this for libvirt I came to better understand the appeal of the gitforge for project infrastructure. I've written about it in detail for libvirt: https://www.redhat.com/archives/libvir-list/2020-March/msg00009.html The distributed SCM model of git democratized access to source control features across contributors. Drive-by contributors have the same level of SCM features as the main project contributors, instead of being second class citizens as they are with a non-distributed SCM. With the hosted Git forges, this concept covers all aspects of project infrastructure. If your project is using GitLab for all its infra needs, then when a user forks the repo they get access to all the same features as the main project. The fork is no longer a second class citizen lacking access to certain parts of the infra. I find this is especially compelling in the area of CI, containers and web hosting. Consider if qemu-web.git is hosted on gitlab, using GitLab CI to generate the static site content, and GitLab pages to host the website. If a user wants to contribute to qemu-web, they can fork the repo and be confident that the CI jobs in their fork are going to work in the exact same way as in the master repo. They can use GitLab Pages to browse the generated content for their fork to see exactly how it will look. This eliminates the big pain point in qemu-web contribution. Many times I was tripped up with the problem of qemu-web.git requiring a version of Jekyll that is incompatible with the version that ships on my distro. > GitLab's Continuous Integration (CI) system provides a powerful way to > perform actions defined in yaml files in qemu.git. This includes > running scripts, builds, publishing build artifacts, etc. We have > already begun using it for automated builds and tests: > https://gitlab.com/qemu-project/qemu/-/blob/master/.gitlab-ci.yml The CI integration has probably been the single best thing about libvirt's switch to GitLab. > GitLab also offers git repo hosting, wikis, issue tracking, and other > features. It is possible to log in using GitHub, Google, or Twitter > single sign-on if you do not want to create another account. As more > open source projects use GitLab it becomes easier for one-time > contributors who will already be familiar with the tools from other > projects. > Here is a full list of GitLab's features: > https://about.gitlab.com/features/ > > GitLab offers the gold/ultimate tier for free to open source projects: > https://about.gitlab.com/solutions/open-source/ > > GitLab itself is open source and can be self-hosted if we decide to > leave in the future. NB self-hosting would reduce features compared to gitlab.com unless you paid for the enterprise edition. You'd also have to bring your own infra for running CI jobs which quickly gets very expensive as freedesktop.org admins have learnt the hard way. The CI resources provided by gitlab.com are incredibly generous. > With this in mind I propose moving qemu.org infrastructure to GitLab > incrementally. This needs to be done carefully to avoid disruption and > only where GitLab meets the requirements. The QEMU project will > continue to have access to cloud hosting for running custom > infrastructure or adding runners to GitLab CI to improve CI > performance. > > The following infrastructure components can be considered for GitLab > migration: > > 1. qemu-web.git static site generation. GitLab CI/CD can build the > static website on each qemu-web.git commit and publish the HTML > artifacts. This is an easy win. it should be especially simple for qemu, compared to libvirt, since qemu.org already used separate sub-domains for other content like git & downloads. > 2. wiki.qemu.org is a MediaWiki instance. Account creation is a hurdle > to one-time or new contributors. It is unclear whether GitLab's wiki > is expressive enough for a lossless conversion of the existing QEMU > wiki. Any volunteers interested in evaluating the wiki migration would > be appreciated. Yeah, this is a potentially big piece of work. We didn't finish this in libvirt either. Looking at the libvirt mediawiki though, I decided not todo a straight export/import of all content. A bunch of content was horribly out of date and needed deleting as it was irrelevant to anyone, or outright misleading wrt current state of the world. A lot of content on the wiki really only existed in the wiki, because contributing to our libvirt.org website was harder than it needed to be. We switched the main website over to use RST instead of HTML so authoring content is much closer to what you'd experience with the wiki markup. The wiki became a burden for contributors as due to spam attacks we can't allow anonymous access, and need manual account creation as with QEMU wiki. Every page on the main website now has a link "edit this page" that takes you straight to the source file in the gitlab.com UI, where you can interactively edit the text in your fork and then submit a merge request with the result. The wiki also suffered from the content not having any level of review. While the intentions of many contributors were good, the results often left alot to be desired. So overall we decided that while GitLab does offer a wiki, we are not intending to use it. Instead we're going to aim to turn the subset of wiki content that is genuinely useful into pages on the main website. IOW, I'd encourage whoever tackles the wiki to look at the content with a critical eye and decide whether it really belongs in the wiki at all. Considering: https://wiki.qemu.org/index.php?title=Special:AllPages The Planning / Changes pages for each release are probably reasonable as wiki pages. The Interships/ProjectIdeas pages would possibly be better off as GitLab Issue tickets, tagged with a "interships" label. That way when a project starts you can have the issue to track the work right from the start. The Documentation/Platforms content arguably should be part of the main qemu.git docs. Many of the feature pages are probably better as part of the formal QEMU documentation too. > 3. Git repo hosting is a core feature of GitLab and we already have a > qemu.git mirror. Hosting the repos on GitLab reduces the need for > qemu.org ssh access. With libvirt we had long used both gitlab.com/libvirt and github.com/libvirt as read-only mirrors of the master libvirt.org/git/ repos. When we formally adopted gitlab as primary infrastructure, we simply flipped the mirroring processes around. So now libvirt.org/git is a read-only mirror, and gitlab is the master. Gitlab has built-in support to push to github.com as a mirror which is useful. So turning git.qemu.org into a readonly mirror would ease the transition to avoid breaking any scripts / workflows relying on that URL in the short term. > 4. The QEMU release process can be moved to CI/CD so that publishing > stable releases and release candidates is less dependent on one > committer's machine or scripts. > > 5. Issue tracking. Launchpad more or less works, but the login always > bothers me. If we move git repo hosting then it makes sense to do > issue tracking on GitLab too. The big thing that always bothers me about launchpad is how easy it is to get confused between issues for QEMU upstream and issues for legacy releases in Ubuntu distro. This is the curse of using a distro issue tracker - libvirt had the same problem with its use of Red Hat bugzilla. Integrated issue tracking alongside the primary git repo makes a lot of sense. Once you declare gitlab.com the primary repo, contributors will immediately look to gitlab for the issue tracking too whether you tell them to or not. The gitlab issue tracker is fairly simple especially compared to the complexity horror of red hat bugzilla, so it was a massive win for libvirt to switch away from bugzilla for issues. Probably not such a big difference compared to launchpad in terms of complexity. The little things though like magic git commit messages to close issues are nice though. There is a question of what todo with existing bugs in launchpad. Essentially three choices 1. Move all the open bugs to gitlab 2. Move some relevant bugs to gitlab, but close outdated ones 3. Leave existing launchpad bugs but don't allow new ones filed So far for libvirt we did option 3 there, with a little bit of option 2. > There is a snowball effect where the experience is improved the more > GitLab features we use, so I hope that most of these migrations will > be possible. > > Next steps: > * If you have an interest in one or more of these infrastructure > components, please join the discussion. > * If there are no volunteers for an infrastructure component I'll > slowly work my way through evaluating GitLab and propose migrations The experiance of moving infra to gitlab.com has been quite positive for libvirt, and me in particular as the single point of failure for much of libvirt's old infrastructure. I think it will be beneficial for QEMU too, for many of the reasons I mention above and in the email link I quoted. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
