On 05/07/18 13:51, Andrew Cooper wrote: > On 05/07/18 12:18, George Dunlap wrote: >> >>>> Another potential problems showed up last week: OSSTEST is using the >>>> Debian servers for doing the basic installation. A change there (e.g. >>>> a new point release) will block tests. I'd prefer to have a local cache >>>> of the last known good set of *.deb files to be used especially for the >>>> branched Xen versions. This would rule out remote problems for releases. >>>> >>>> This is again something which we should definitely look at. >>> This was bad luck. This kind of update happens about 3-4 times a >>> year. It does break everything, leading to a delay of a day or two, >>> but the fix is straightforward. >>> >>> Obviously this is not ideal but the solutions are nontrivial. It is >>> not really possible to "have a local cache of the last known good set >>> of *.deb files" without knowing what that subset should be; that would >>> require an edifice to track what is used, or some manual configuration >>> which would probably break. Alternatively we could run a complete >>> mirror but that is a *lot* of space and bandwidth, most of which would >>> be unused. >>> >>> I think the right approach is probably to switch from using d-i for >>> host installs, to something like FAI. That would be faster as well. >>> However that amouns to reengineering the way osstest does host >>> installs; it would also leave us maintaining an additional way to do >>> host installs, since we would still want to be able to *test* d-i >>> operation as a guest. >> What I think would be ideal is a way to take ‘snapshots’ of different states >> of setup for various hosts and revert to them. There’s absolutely no reason >> to do a full install of a host every osstest run, when that install happens >> 1) before we even install Xen, and 2) should be nearly identical each time. >> We should be able to install a host, take a snapshot of the “clean” install, >> then do the build prep, take a snapshot of that, and then simply revert to >> one or both of those (assuming build requirements haven’t changed in the >> mean time) whenever necessary. Re-generating these snapshots once per week >> per host should be plenty, and sounds like it would massively improve the >> current throughput. >> >> I’d like to propose the idea also that we try to find a more efficient way >> of testing guest functionality than doing a guest install. I understand >> it’s a natural way to test a reasonable range of functionality, but >> particularly for Windows guests, my impression is that it’s very slow; there >> must be a way to make a test that would have similar coverage but be able to >> be completed with a pre-installed snapshot, in only a few minutes. > > We've had similar discussions in XenServer. That idea is superficially > attractive but actually makes things worse, because it now means that > filesystem clone/snapshot is now in the mix of things which can go wrong. > > Particularly with OSSTest testing mainline kernels, rather than distro > stable kernels, the chances of finding filesystem bugs grows > substantially, and the complexity of diagnosing an issue is outside of > our area of expertise.
But are really all tests required to use a freshly installed mainline kernel? Can't we e.g. do a guest install test once a new kernel is released and clone the resulting guest disk image for other tests (after shutting down the guest, of course)? Same applies to the host: the base system (without the to be tested component like qemu, xen, or whatever) could be installed just by cloning a disk/partition/logical volume. Each image would run through the stages new->staging->stable: - Each time a component is released an image is based on (e.g. a new mainline kernel) a new image is created by installing it. In case this succeeds, the image is moved to the staging area. - The images in the staging area are tested using known stable components in order to test the image, not the test-components. In case of all tests succeeded the image is moved to the stable area. - The stable images are used to test components from staging. In case of success the related components can be pushed to stable/master. Juergen _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
