On Mon, 2023-07-10 at 16:12 +0200, Heinrich Schuchardt wrote: > On 10.07.23 00:29, Steve Langasek wrote: > > On Sun, Jul 09, 2023 at 04:28:42AM +0200, Heinrich Schuchardt wrote: > > > > > Benjamin Drung <bdr...@ubuntu.com> schrieb am Sa., 8. Juli 2023, > > > > > 02:19: > > > > > > > > Hi all, > > > > > > > > a year ago we changed the default compression and level for the > > > > > > initramfs to zstd -1. This fixed the very slow creation times on > > > > > > development boards (see bug #1958148), but that leads to bigger > > > > > > initramfs sizes that triggered other bugs (like bug #1842320). > > > > > > Big initramfs sizes can also fill up small sized /boot partitions > > > > > > easily > > > > > > (grooming the 850 initramfs-tools bugs revealed several such > > > > > > reports). > > > > > > > > Using xz -9 would give very good compression, but it takes very long > > > > > > (especially on slow development boards) and a lot of memory (good > > > > > > luck > > > > > > on Raspberry Pis with small memory like Pi Zeros). > > > > > > > > I propose following approach to address the drawback: Create cpio > > > > > > archives (compressed with xz -9) for the kernel modules and firmware > > > > > > files when building the kernel/firmware Debian package. Then ship > > > > > > those > > > > > > cpio archives in the package (or in a separate binary package). > > > > > > Then the > > > > > > CPU load it put on the builders. The cpio archives would contain the > > > > > > modules for MODULES=most. > > > > > > > > mkinitramfs will then look for those cpio archives and uses those in > > > > > > case they are present. Such a initramfs would look like this: > > > > > > > > * AMD/Intel microcode cpio archive (on amd64) > > > > > > * main cpio archive compressed with zstd -1 > > > > > > * kernel modules from the Debian package compressed with xz -9 > > > > > > * firmware files from the Debian package compressed with xz -9 > > > > > > > > After working on initramfs-tools as part my day job, my fingers were > > > > > > itching and I had to create a quick and dirty draft in my free night > > > > > > time. You can find the result of the last two hours in [1]. This > > > > > > draft > > > > > > has a mkinitramfs-kernel script that creates a cpio archive > > > > > > containing > > > > > > the kernel modules and firmware (that needs to be split later on). > > > > > > > > The lunar test result on my AMD Ryzen 7 5700G look promising: > > > > > > Building > > > > > > 6.2.0-24-generic-modules-most.cpio.xz takes around 90 seconds and is > > > > > > 54.9 MiB in size. Creating the initramfs speeds up from around 8.7 > > > > > > seconds to 3.5 seconds (saves 60 %). The size reduces from 133.1 > > > > > > MiB to > > > > > > 80.7 MiB (saves 39.4 %). So the boot needs 52.4 MiB less, but > > > > > > /lib/modules need 54.9 MiB for the cpio archive. > > > > > > > > The drawback is that building the kernel would take longer, the > > > > > > package > > > > > > takes more space on the archive and mirrors, and downloading them > > > > > > could > > > > > > take longer on slow connections. > > > > > > > > Implementing my proposal would be relative easy for > > > > > > initramfs-tools, but > > > > > > would mean some work for the kernel team. > > > > > > > > What do you think? > > > > > Will the user still be able to add further modules and will machine > > > specific > > > configuration files (e.g. for booting from iSCSI) still be included into > > > the > > > initrd? > > > > I think a robust implementation of this on the initramfs-tools side looks > > like: > > > > - identify all the contents that belong in the initramfs > > - among those contents, find all zstd-compressed files, if any, and store > > them in an uncompressed initramfs > > - put the rest of the contents in a compressed initramfs > > > > This would be compatible with kernel packages whether they are changed to > > ship zstd-compressed modules or not and allow for a smooth transition. > > > > > > I built a kernel with CONFIG_MODULE_COMPRESS_ZSTD=y. > > update-initramfs cannot find a module specified in > /etc/initramfs-tools/conf.d/modules_list.conf due to the .zstd extension.
That should work with the current initramfs-tools 0.142ubuntu7. This version only ignores zstd compressed firmware file (and uncompresses xz and zst kernel modules before creating the initramfs). > Modules are uncompressed before being added to the initrd. > > We would need an updated initramfs-tools package for evaluating that path. Here you have initramfs-tools 0.142ubuntu7bd2 for mantic in my PPA https://launchpad.net/~bdrung/+archive/ubuntu/ppa for testing. -- Benjamin Drung Debian & Ubuntu Developer -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
