On Thu, Mar 14, 2019 at 06:22:44PM +0000, Peter Maydell wrote: > On Thu, 14 Mar 2019 at 15:57, Alex Bennée <alex.ben...@linaro.org> wrote: > > Testing in the Cloud > > ==================== > > > > After BuildBot went out-of-service we have been relying heavily on Travis > > as our primary CI platform. This has been creaking somewhat under the > > strain and while we have a large test matrix its coverage is fairly > > Ubuntu/x86 centric. However in recent months we've expanded and we now > > have: > > > > - Shippable, cross compilers - catches a lot of 32/64 bit isms > > - Cirrus, FreeBSD and MacOS builds > > - GitLab, Alternative x86/Debian - iotests > > Are any of these capable of replacing my ad-hoc collection > of build test systems for testing merges ? I would quite like > to be able to do that, because it would make it easier for > other people to take over the process of handling pull requests > when I'm away. > > I think the main requirements for that would be: > * covers full range of hosts[*] > * can be asked to do a test build of a merge before > I push it to master > * reliably completes all builds within say 90 minutes > of being asked to start
Out of all of the systems above, I would personally have a strong preference for GitLab simply because it is actually based on open source software. Putting a critical part of QEMU's workflow on to a closed source service whose infrastructure or terms of service could change at any time, feels risky to me. With GitLab in the worst case we can spin up an instance on our own hardware to run our test stack, especially if we are already providing our own test runners. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
