Hi Alistair, Fam,
On 05/21/2018 12:16 AM, Fam Zheng wrote:
> On Fri, 05/18 11:34, Alistair Francis wrote:
>> Avocado is not trivial to setup on non-Fedora systems. To simplfying
>> future testing add a docker test image that runs Avocado tests.
Can you add an entry in the "make docker" help menu?
>>
>> Signed-off-by: Alistair Francis <alistair.fran...@wdc.com>
>> ---
>> v2:
>> - Add a seperate fedora-avocado Docker image
>> - Move the avocado vt-bootstrap into the Docker file
>>
>> tests/docker/Makefile.include | 1 +
>> .../docker/dockerfiles/fedora-avocado.docker | 25 +++++++++++++++++
>> tests/docker/test-avocado | 28 +++++++++++++++++++
>> 3 files changed, 54 insertions(+)
>> create mode 100644 tests/docker/dockerfiles/fedora-avocado.docker
>> create mode 100755 tests/docker/test-avocado
>>
>> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
>> index ef1a3e62eb..0e3d108dde 100644
>> --- a/tests/docker/Makefile.include
>> +++ b/tests/docker/Makefile.include
>> @@ -60,6 +60,7 @@ docker-image-debian-ppc64el-cross: docker-image-debian9
>> docker-image-debian-s390x-cross: docker-image-debian9
>> docker-image-debian-win32-cross: docker-image-debian8-mxe
>> docker-image-debian-win64-cross: docker-image-debian8-mxe
>> +docker-image-fedora-avocado: docker-image-fedora
>> docker-image-travis: NOUSER=1
>>
>> # Expand all the pre-requistes for each docker image and test combination
>> diff --git a/tests/docker/dockerfiles/fedora-avocado.docker
>> b/tests/docker/dockerfiles/fedora-avocado.docker
>> new file mode 100644
>> index 0000000000..55b19eebbf
>> --- /dev/null
>> +++ b/tests/docker/dockerfiles/fedora-avocado.docker
>> @@ -0,0 +1,25 @@
>> +FROM qemu:fedora
>> +
>> +ENV PACKAGES \
>> + libvirt-devel \
>> + nc \
>> + python-avocado \
>> + python2-devel python3-devel \
>> + qemu-kvm \
>> + tcpdump \
>> + xz
>> +ENV PIP_PACKAGES \
>> + avocado-qemu \
>> + avocado-framework-plugin-runner-remote \
>> + avocado-framework-plugin-runner-vm \
>> + avocado-framework-plugin-vt
>> +
>> +ENV QEMU_CONFIGURE_OPTS --python=/usr/bin/python3
>
> I think this is inherited from qemu:fedora, no?
Yes.
>
>> +
>> +RUN dnf install -y $PACKAGES
>> +RUN pip install $PIP_PACKAGES
>> +RUN avocado vt-bootstrap --yes-to-all --vt-type qemu
>> +
>> +RUN rpm -q $PACKAGES | sort > /packages.txt
>
> Can you keep the parent image's list with ">>" or appending to the old
> $PACKAGES
> in the above "ENV" directive?
Appending looks cleaner to me.
>
>> +
>> +ENV FEATURES mingw clang pyyaml asan avocado
>
> Similarly, is it possible to append to the parent list instead of overriding?
>
>> diff --git a/tests/docker/test-avocado b/tests/docker/test-avocado
>> new file mode 100755
>> index 0000000000..40474db2ce
>> --- /dev/null
>> +++ b/tests/docker/test-avocado
>> @@ -0,0 +1,28 @@
>> +#!/bin/bash -e
>> +#
>> +# Avocado tests on Fedora, as these are a real pain on Debian systems
>
> Shouldn't pip packages work just well on Debian too? What are the pain?
> (Cc'ing Cleber who may want to know this).
Avocado isn't packaged (yet?) on Debian.
>
> Fam
>
>> +#
>> +# Copyright (c) 2018 Western Digital.
>> +#
>> +# Authors:
>> +# Alistair Francis <alistair.fran...@wdc.com>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2
>> +# or (at your option) any later version. See the COPYING file in
>> +# the top-level directory.
>> +#
>> +# Run this test: NOUSER=1 make docker-test-avocado@fedora-avocado
>> +
>> +. common.rc
>> +
>> +requires avocado
>> +
>> +cd "$BUILD_DIR"
>> +
>> +DEF_TARGET_LIST="x86_64-softmmu"
>> +TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
>> +build_qemu
>> +install_qemu
>> +
>> +export PATH="${PATH}:$(pwd)"
>> +avocado run boot --vt-qemu-bin ./x86_64-softmmu/qemu-system-x86_64
This failed when testing (I suppose due to too old corporate proxy...):
Step 7/11 : RUN avocado vt-bootstrap --yes-to-all --vt-type qemu
---> Running in 008e494971c7
[...]
8 - Verifying (and possibly downloading) guest image
Verifying expected SHA1 sum from
http://avocado-project.org/data/assets/jeos/27/SHA1SUM_JEOS_27_64
Failed to get SHA1 from file: HTTP Error 403: Forbidden file type or
location: http://avocado-project.org/data/assets/jeos/27/SHA1SUM_JEOS_27_64
File /var/lib/avocado/data/avocado-vt/images/jeos-27-x86_64.qcow2.xz not
found
Check your internet connection: HTTP Error 403: Forbidden file type or
location: http://avocado-project.org/data/assets/jeos/27/jeos-27-64.qcow2.xz
Uncompressing
/var/lib/avocado/data/avocado-vt/images/jeos-27-x86_64.qcow2.xz ->
/var/lib/avocado/data/avocado-vt/images/jeos-27-x86_64.qcow2
Bootstrap command failed
Command: xz -cd
/var/lib/avocado/data/avocado-vt/images/jeos-27-x86_64.qcow2.xz >
/var/lib/avocado/data/avocado-vt/images/jeos-27-x86_64.qcow2
stderr output:
xz: /var/lib/avocado/data/avocado-vt/images/jeos-27-x86_64.qcow2.xz:
File format not recognized
The command '/bin/sh -c avocado vt-bootstrap --yes-to-all --vt-type
qemu' returned a non-zero code: 1
Traceback (most recent call last):
File "./tests/docker/docker.py", line 407, in <module>
sys.exit(main())
File "./tests/docker/docker.py", line 404, in main
return args.cmdobj.run(args, argv)
File "./tests/docker/docker.py", line 326, in run
extra_files_cksum=cksum)
File "./tests/docker/docker.py", line 209, in build_image
quiet=quiet)
File "./tests/docker/docker.py", line 147, in _do_check
return subprocess.check_call(self._command + cmd, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 190, in check_call
raise CalledProcessError(retcode, cmd)
make: *** [tests/docker/Makefile.include:37:
docker-image-fedora-avocado] Error 1
We said we should use some userdir volume to cache the data/assets/,
1/ to avoid to keep downloading this file again and again,
2/ to share it between different docker instances,
3/ so I can download it via another way and put it there (proxy).
BTW Fam (unrelated) when building the intermediary Fedora image I hit:
[...]
(562/563): man-pages-4.12-1.fc27.noarch.rpm 270 kB/s | 5.9 MB
00:22
(563/563): glibc-common-2.26-27.fc27.x86_64.rpm 62 kB/s | 789 kB
00:12
--------------------------------------------------------------------------------
Total 10 kB/s | 425 MB
709:02
Running transaction check
Transaction check succeeded.
[...]
Error: Transaction check error:
installing package glibc-common-2.26-27.fc27.x86_64 needs 4MB on the /
filesystem
installing package dconf-0.26.1-1.fc27.x86_64 needs 2016MB on the /
filesystem
installing package man-pages-4.12-1.fc27.noarch needs 2018MB on the /
filesystem
Error Summary
-------------
Disk Requirements:
At least 2018MB more space needed on the / filesystem.
subprocess.CalledProcessError: Command '['docker', 'build', '-t',
'qemu:fedora', '-f', '/tmp/docker_buildd0ajZ0/tmp1jgnS6.docker',
'/tmp/docker_buildd0ajZ0']' returned non-zero exit status 1
make: *** [tests/docker/Makefile.include:37: docker-image-fedora] Error 1
I wonder if we should add an estimated size of image, to avoid (like in
my case, wait another night to test this due to my ****** internet
connection).
Regards,
Phil.
