Daniel P. Berrangé <berra...@redhat.com> writes:
> On Tue, Oct 04, 2022 at 02:01:12PM +0100, Alex Bennée wrote:
>> From: Paolo Bonzini <pbonz...@redhat.com>
>>
>> s390-ccw remains a bit more complex, because the -march=z900 test is done
>> only for the native cross compiler. Otherwise, all that is needed is
>> to pass the (now mandatory) target argument to write_target_makefile.
>>
>> Signed-off-by: Paolo Bonzini <pbonz...@redhat.com>
>> Signed-off-by: Alex Bennée <alex.ben...@linaro.org>
>> Message-Id: <20220929114231.583801-29-alex.ben...@linaro.org>
>
> I'm not at all convinced this change was/is a good idea.
>
> First of all, it causes 'make' to now download about 1 GB of
> container images
>
> $ ./configure --target-list=x86_64-softmmu
> $ make
> ...snip...
> BUILD debian-powerpc-test-cross
> Trying to pull
> registry.gitlab.com/qemu-project/qemu/qemu/debian-powerpc-test-cross:latest...
> Getting image source signatures
> Copying blob 2a205c8a1d36 [=>------------------------------------] 12.4MiB
> / 257.2MiB
>
> ...
> ...snip...
>
> Despite downloading this image, it then proceeded to rebuild the
> image from scratch, requiring another few 100MBs of downloads
> of dpkgs. This time the download was without progress information
> until it entirely failed due to a dead Debia mirror server, needing
> a retry.
>
> It then went on to download an s390x image which seems to have
> two layers, each with 360 MB.
>
> BUILD debian-s390x-cross
> Trying to pull
> registry.gitlab.com/qemu-project/qemu/qemu/debian-s390x-cross:latest...
> Getting image source signatures
> Copying blob fc8d65e34cd5 [>-------------------------------------] 12.0MiB /
> 360.2MiB
> Copying blob bd159e379b3b skipped: already exists
> Copying blob 13224e2971af [>-------------------------------------] 12.2MiB /
> 366.5MiB
>
> So overall it was more than 1 GB of downloads when typing 'make'
>
> I wasn't too amuzed by seeing this downloaded data , given that
> I'm usually running off a 4G mobile connection, and it took a
> very long time.
Yikes, sorry I didn't notice that (probably because I always have most
of the containers built).
I was hoping the next set of patches would reduce the total re-build
time to just the mirror operation by dumping docker.py and any caching
that breaks.
> The progress information printed by docker when downloading
> the images splatters all over the output meson displays, when
> doing a parallel make making everything unintelligible.
>
>
> Finally, I had requested only building x86_64, so we shouldn't
> be doing anything related to ppc or s390 at all, but even if
>
> AFAICT, it enables this downloading unconditionally merely by
> having 'docker'/'podman' binaries installed, if you don't
> otherwise have cross compuilers present.
>
> I'd really not want to see any of this stuff downloaded without
> an explicit opt-in choice at configure time.
>
> I'm also a little concerned at what happens if we have to stop
> publishing the containers at registry.gitlab.com in future. Are
> we going to break the default 'make' for existing released QEMU
> tarballs ?
We can easily move the registry around. The aim of this work is to
eventually stop local re-builds for most people.
>
> Generally we've only relied on the gitlab infra for our CI
> testing, so we have been free to change infra or alter the
> way we publish images at any time, without risk of impact on
> the released tarballs.
>
> This isn't a theoretical problem, because GitLab has announced
> their intention to limit storage usage in gitlab.com, and even
> having joined the Open Source Program, our quota is only increased
> from 5 GB to 25 GB. I'd be concerned we're at risk of exceeding
> that 25 GB limit, when they start to enforce it, requiring us to
> move container image host to somewhere else such as quay.io
>
>
>> diff --git a/configure b/configure
>> index c175650eb9..a54e17aca9 100755
>> --- a/configure
>> +++ b/configure
>> @@ -2152,7 +2152,7 @@ probe_target_compiler() {
>> target_ranlib=
>> target_strip=
>> fi
>> - test -n "$target_cc"
>> + test -n "$target_cc" || test -n "$container_image"
>> }
>>
>> write_target_makefile() {
>> @@ -2307,7 +2307,7 @@ if test "$targetos" != "darwin" && test "$targetos" !=
>> "sunos" && \
>> config_mak=pc-bios/optionrom/config.mak
>> echo "# Automatically generated by configure - do not modify" >
>> $config_mak
>> echo "TOPSRC_DIR=$source_path" >> $config_mak
>> - write_target_makefile >> $config_mak
>> + write_target_makefile pc-bios/optionrom/all >> $config_mak
>> fi
>>
>> if test "$softmmu" = yes && probe_target_compiler ppc-softmmu; then
>> @@ -2315,25 +2315,31 @@ if test "$softmmu" = yes && probe_target_compiler
>> ppc-softmmu; then
>> config_mak=pc-bios/vof/config.mak
>> echo "# Automatically generated by configure - do not modify" >
>> $config_mak
>> echo "SRC_DIR=$source_path/pc-bios/vof" >> $config_mak
>> - write_target_makefile >> $config_mak
>> + write_target_makefile pc-bios/vof/all >> $config_mak
>> fi
>>
>> # Only build s390-ccw bios if the compiler has -march=z900 or -march=z10
>> # (which is the lowest architecture level that Clang supports)
>> if test "$softmmu" = yes && probe_target_compiler s390x-softmmu; then
>> - write_c_skeleton
>> - do_compiler "$target_cc" $target_cc_cflags -march=z900 -o $TMPO -c $TMPC
>> - has_z900=$?
>> - if [ $has_z900 = 0 ] || do_compiler "$target_cc" $target_cc_cflags
>> -march=z10 -msoft-float -Werror -o $TMPO -c $TMPC; then
>> - if [ $has_z900 != 0 ]; then
>> - echo "WARNING: Your compiler does not support the z900!"
>> - echo " The s390-ccw bios will only work with guest CPUs >=
>> z10."
>> + got_cross_cc=no
>> + if test -n "$target_cc"; then
>> + write_c_skeleton
>> + do_compiler "$target_cc" $target_cc_cflags -march=z900 -o $TMPO -c $TMPC
>> + has_z900=$?
>> + if [ $has_z900 = 0 ] || do_compiler "$target_cc" $target_cc_cflags
>> -march=z10 -msoft-float -Werror -o $TMPO -c $TMPC; then
>> + if [ $has_z900 != 0 ]; then
>> + echo "WARNING: Your compiler does not support the z900!"
>> + echo " The s390-ccw bios will only work with guest CPUs >=
>> z10."
>> + fi
>> + got_cross_cc=yes
>> fi
>> + fi
>> + if test "$got_cross_cc" = yes || test -n "$container_image"; then
>> roms="$roms pc-bios/s390-ccw"
>> config_mak=pc-bios/s390-ccw/config-host.mak
>> echo "# Automatically generated by configure - do not modify" >
>> $config_mak
>> echo "SRC_PATH=$source_path/pc-bios/s390-ccw" >> $config_mak
>> - write_target_makefile >> $config_mak
>> + write_target_makefile pc-bios/s390-ccw/all >> $config_mak
>> # SLOF is required for building the s390-ccw firmware on s390x,
>> # since it is using the libnet code from SLOF for network booting.
>> git_submodules="${git_submodules} roms/SLOF"
>> @@ -2554,7 +2560,7 @@ for target in $target_list; do
>> ;;
>> esac
>>
>> - if probe_target_compiler $target || test -n "$container_image"; then
>> + if probe_target_compiler $target; then
>> test -n "$container_image" && build_static=y
>> mkdir -p "tests/tcg/$target"
>> config_target_mak=tests/tcg/$target/config-target.mak
>> --
>> 2.34.1
>>
>>
>
> With regards,
> Daniel
--
Alex Bennée
