On Thu, Apr 03, 2025 at 12:39:11PM +0100, Andrew Cooper wrote: > On 03/04/2025 12:29 pm, Jan Beulich wrote: > > On 03.04.2025 13:04, Marek Marczykowski-Górecki wrote: > >> It fails on larger initramfs (~250MB one) and sometimes even smaller > >> depending on memory size/memory map, let Linux do it. > > Iirc grub only unpacks gzip-ed modules, so wouldn't a yet better approach > > be to use a better compressing algorithm, which simply as a side effect > > would keep grub from decompressing it, while at the same time moving > > farther away from any critical boundaries? > > Yes and no. > > This is going to change anyway when I (or a delgee) moves initrd > generation from the test step itself into the test artefacts repo.
Out of curiosity, I tried this: https://gitlab.com/xen-project/people/marmarek/xen/-/commit/51b5fde816806206e6e0c72ec0146c5ab3da4351 pipeline: https://gitlab.com/xen-project/people/marmarek/xen/-/pipelines/1753304751 It avoids repacking base alpine initrd, but it still needs to pack per-job stuff (test script, but also toolstack). The gains are not great, that step still takes over a minute. The overall job time went down by just 40-50s (for example from 6m50s to 6m03s). This isn't very surprising, because the base initrd.cpio.gz is 60MB and the final one is over 200MB, so the part that was needlessly re-compressed is about 30%. The numbers checks out, as the (debug) build artifacts zip is about 150MB. One further idea would be to produce cpio.gz archive as part of the build artifact, to avoid zip (used by gitlab for artifacts) -> cpio.gz recompression. But that either makes build artifacts bigger for every job, or need setting a variable for jobs that are used later for tests. Unless we decide to store toolstack _only_ as cpio.gz in artifacts? I tried the latter idea here: https://gitlab.com/xen-project/people/marmarek/xen/-/commit/a4774213d83ca05c0b0b0fff80e49cf3aa32b67b pipeline: https://gitlab.com/xen-project/people/marmarek/xen/-/pipelines/1753312631 This helped quite a bit more. The 1m of compressing initrd is gone now. Compressing remaining test-specific files and concatenating it all together takes 2-3s. Now that 6m50s job is down to 3m40s. I'm not exactly sure where that extra minute went (maybe related to number of files in the build artifact? Fetching/extracting artifacts seems to be faster a bit), but those times are not very consistent anyway, so it could be also just random fluctuation. Looking further at the test log, some time is used for fetching artifacts. I've looked into gitlab docs if those can be cached on the runner, but I haven't found a solution that would really help (you can cache arbitrary paths, including artifacts, but it doesn't prevent re-downloading them and unpacking over the cached version...). So, I'll stop here, polish the above patches and submit them in v2 of this series. -- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab
signature.asc
Description: PGP signature
