On Tue, Mar 12, 2019 at 08:30:02PM +0100, Laszlo Ersek wrote: > On 03/12/19 16:59, Daniel P. Berrangé wrote: > > On Tue, Mar 12, 2019 at 04:40:45PM +0100, Laszlo Ersek wrote: > >> On 03/11/19 11:35, Daniel P. Berrangé wrote: > >>> On Sun, Mar 10, 2019 at 12:21:55PM +0100, Philippe Mathieu-Daudé wrote: > >>>> On 3/10/19 4:56 AM, Michael S. Tsirkin wrote: > >>>>> On Sat, Mar 09, 2019 at 01:48:16AM +0100, Laszlo Ersek wrote: > >>>>>> Repo: https://github.com/lersek/qemu.git > >>>>>> Branch: edk2_build > >>>>>> > >>>>>> This series advances the roms/edk2 submodule to the "edk2-stable201903" > >>>>>> release, and builds and captures platform firmware binaries from that > >>>>>> release. At this point they are meant to be used by both end-users and > >>>>>> by Igor's ACPI unit tests in qtest ("make check"). > >>>>>> > >>>>>> Previous discussion: > >>>>>> > >>>>>> [Qemu-devel] bundling edk2 platform firmware images with QEMU > >>>>>> > >>>>>> 80f0bae3-e79a-bb68-04c4-1c9c684d95b8@redhat.com">http://mid.mail-archive.com/80f0bae3-e79a-bb68-04c4-1c9c684d95b8@redhat.com > >>>>>> https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg02601.html > >>>> > >>>> There David raised a concern about "[adding] ~206 MB of binaries to the > >>>> pc-bios directory". I'm also worried. > >>>> > >>>> GitHub kindly suggest to use git-lfs. It is an extra dependency I'd > >>>> rather strongly avoid (because we support a wide range of host OS, each > >>>> using a wide types of filesystems). > >>>> > >>>> What about storing those binaries on a file server (http/ftp) altogether > >>>> with a file containing its hashed digest (SHA1/SHA256)? Then we already > >>>> have all the required tools to fetch and verify those blob roms with the > >>>> build system. > >>>> Or we could store the hashes in the QEMU repository too. > >>> > >>> A simple approach would simply be to 'xz' compress all the edk*.fd > >>> files but still store them in git. They're already opaque binary > >>> files, so replacing one binary format with another binary format > >>> is no big deal IMHO > >>> > >>> $ ls -alsh edk2-* > >>> 1.1M -rw-rw-r--. 1 berrange berrange 1.1M Mar 11 10:29 > >>> edk2-aarch64-code.fd.xz > >>> 2.1M -rw-rw-r--. 1 berrange berrange 64M Mar 11 10:29 edk2-arm-code.fd > >>> 772K -rw-rw-r--. 1 berrange berrange 64M Mar 11 10:29 edk2-arm-vars.fd > >>> 3.5M -rw-rw-r--. 1 berrange berrange 3.5M Mar 11 10:29 edk2-i386-code.fd > >>> 3.5M -rw-rw-r--. 1 berrange berrange 3.5M Mar 11 10:29 > >>> edk2-i386-secure-code.fd > >>> 528K -rw-rw-r--. 1 berrange berrange 528K Mar 11 10:29 edk2-i386-vars.fd > >>> 12K -rw-rw-r--. 1 berrange berrange 11K Mar 11 10:29 edk2-licenses.txt > >>> 3.5M -rw-rw-r--. 1 berrange berrange 3.5M Mar 11 10:29 edk2-x86_64-code.fd > >>> 3.5M -rw-rw-r--. 1 berrange berrange 3.5M Mar 11 10:29 > >>> edk2-x86_64-secure-code.fd > >>> > >>> This gives a 50% disk space saving over the sparse size: > >>> > >>> $ ls -alsh edk2-* > >>> 1.1M -rw-rw-r--. 1 berrange berrange 1.1M Mar 11 10:29 > >>> edk2-aarch64-code.fd.xz > >>> 1.1M -rw-rw-r--. 1 berrange berrange 1.1M Mar 11 10:29 edk2-arm-code.fd.xz > >>> 12K -rw-rw-r--. 1 berrange berrange 9.8K Mar 11 10:29 edk2-arm-vars.fd.xz > >>> 1.6M -rw-rw-r--. 1 berrange berrange 1.6M Mar 11 10:29 > >>> edk2-i386-code.fd.xz > >>> 1.8M -rw-rw-r--. 1 berrange berrange 1.8M Mar 11 10:29 > >>> edk2-i386-secure-code.fd.xz > >>> 4.0K -rw-rw-r--. 1 berrange berrange 320 Mar 11 10:29 > >>> edk2-i386-vars.fd.xz > >>> 12K -rw-rw-r--. 1 berrange berrange 11K Mar 11 10:29 edk2-licenses.txt > >>> 1.6M -rw-rw-r--. 1 berrange berrange 1.6M Mar 11 10:29 > >>> edk2-x86_64-code.fd.xz > >>> 1.9M -rw-rw-r--. 1 berrange berrange 1.9M Mar 11 10:29 > >>> edk2-x86_64-secure-code.fd.xz > >>> > >>> So about 9 MB compressed, instead of 20MB for the uncompressed sparse > >>> files, which is on a par with the existing ROM sizes. > >>> > >>> We would then need a "make" rule that runs unxz to "build" the firmware > >>> files. Wouldn't need to be more complicated that this: > >>> > >>> edk2-%.fd: edk2-%.fd.xz > >>> unzx -c $< > $@ > >>> > >>> CLEANFILES += $(wildcard edk2*.fd) > >> > >> The problem with this idea is that such *.xz files are not directly > >> consumeable by Igor's ACPI regression test cases in qtest. The above > >> approach is suitable for "make install" only, not for "make check", AIUI. > > > > I was suggesting that this rule to uncompress is wired into the > > default "make" target actually. > > Ah, I see. > > We might have to mark these "side artifacts" as .SECONDARY then, lest > make remove them in the end. > > https://www.gnu.org/software/make/manual/make.html#Chained-Rules > > Not entirely sure yet, I'd have to see in practice. > > > Probably by just marking them > > as generated files, so they are always built. This will make > > it work for tests, as well as developers running from the source > > tree. > > I'm quite lost in the top level Makefile -- this idea is more complex > than just extending INSTALL_BLOBS / BLOBS. > > What target should I make dependent on the uncompressed fd files so that > "make" process them for both "make" (which is "make all" I guess) and > "make check"?
I think it is sufficient to edit the main Makefile and add the list of UEFI firmwares to the GENERATED_FILES variable. Anything listed in $GENERATED_FILES is generated before any of the other make rules run. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
