Hi, On 2024-06-17 08:34, Johannes Schauer Marin Rodrigues wrote: > Quoting Francesco Poli (wintermute) (2024-06-16 19:09:08) >> But, the allocated size has significantly grown: >> >> $ cd ~/.cache/sbuild/ >> $ ls -altrFs --si >> total 4.4G >> 4.1k drwx------ 37 $USER $USER 4.1k May 4 16:03 ../ >> 4.1k drwxrwx--- 2 $USER $USER 4.1k May 13 23:19 build/ >> 3.6G -rw-rw---- 1 $USER $USER 27G Jun 9 22:00 >> OLD_unstable-autopkgtest-amd64.img >> 4.1k drwxrwx--- 3 $USER $USER 4.1k Jun 16 18:26 ./ >> 832M -rw-rw---- 1 $USER $USER 27G Jun 16 18:26 >> unstable-autopkgtest-amd64.img >> >> Now the allocated size is 832 MB, instead of 705 MB !!!
In this particular case, one factor would be that the update caused new APT lists to be downloaded, which have tens of MB. >> But why does the allocated size increase? >> Maybe there's something about sparse file support that I do not >> fully understand. >> >> Is there anything that can be done inside sbuild-qemu-update to prevent >> the allocated size from growing indefinitely? >> Apart from periodically regenerating the image from scratch, I mean... > > as you suspected this is because of how sparse files work. Whenever you > upgrade > something in your image, data gets deleted and new data gets added. The > filesystem driver in the kernel does not zero-out those parts that it deletes > and even if it would, qemu has no idea which blocks of the underlying image > file it should now mark "sparse". Exactly this. One has to differentiate between what goes on in the guest (file deletion from ext4) and what QEMU sees (just blocks being used up). I guess clever stuff could be done but honestly, it's probably simpler to occasionally regenerate an image. A hacky solution would be to use the --snapshot option to sbuild-qemu-update on first run. In future runs, you could reset the image to that snapshot using qemu-img. That would be a tradeoff though as with time, updates would take longer and longer. > One tool that should reduce size again is e2image from e2fsprogs: > > $ e2image -rap old.img new.img > > But this requires copying the actual file data. I didn't try it out, but there > is also the "discard" extended option of e2fsck: > > $ e2fsck -E discard your.img > > Lastly, I do not know if the zerofree tool has support for sparse files? Maybe > try running it on your FS and see what happens. :) Best, Christian
