Ani Sinha <a...@anisinha.ca> writes:
> Please see the README file in patch 9 for more details.
> Sample runs are as follows:
>
> Passed tests:
>
> $ ./tests/venv/bin/avocado run -t acpi tests/avocado
> Fetching asset from
> tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> JOB ID : e95c7413e996bfb59389839e5ca5105464ef098f
> JOB LOG :
> /home/anisinha/avocado/job-results/job-2022-10-14T19.15-e95c741/job.log
> (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS
> (33.08 s)
> RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 |
> CANCEL 0
> JOB TIME : 39.18 s
Hmm something went wrong:
➜ ./tests/venv/bin/avocado run -t acpi tests/avocado
Fetching asset from
tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
JOB ID : 08b6d26bf52a22ccbe1908b14d5b7b27f76ad7fa
JOB LOG :
/home/alex/avocado/job-results/job-2022-10-18T16.29-08b6d26/job.log
(1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: ERROR:
Error while generating the bits iso. Pass V=1 in the environment to get more
details. Command '[
'/var/tmp/acpi-bits-yy4mfxrz.tmp/grub-inst-x86_64-efi/bin/grub-mkrescue',
'-o', '/var/tmp/acpi-bits-yy4mfxrz.tmp/bits-2020.iso',
'/var/tmp/acpi-bits-yy4mfxrz.t... (7.21 s)
RESULTS : PASS 0 | ERROR 1 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 |
CANCEL 0
JOB TIME : 7.79 s
have we added another binary dependency?
>
>
> $ ./tests/venv/bin/avocado run ./tests/avocado/acpi-bits.py
> Fetching asset from
> ./tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> JOB ID : c6e588594bceb2c5fdbdf45095559c58e4030f9c
> JOB LOG :
> /home/anisinha/avocado/job-results/job-2022-10-14T19.13-c6e5885/job.log
> (1/1) ./tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS
> (32.29 s)
> RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 |
> CANCEL 0
> JOB TIME : 39.27 s
>
> When PSS tests are enabled:
>
> $ git diff
> diff --git a/tests/avocado/acpi-bits/bits-tests/testacpi.py2
> b/tests/avocado/acpi-bits/bits-tests/testacpi.py2
> index f818a9cce6..18dc818d62 100644
> --- a/tests/avocado/acpi-bits/bits-tests/testacpi.py2
> +++ b/tests/avocado/acpi-bits/bits-tests/testacpi.py2
> @@ -40,8 +40,8 @@ import time
>
> def register_tests():
> testsuite.add_test("ACPI _MAT (Multiple APIC Table Entry) under
> Processor objects", test_mat, submenu="ACPI Tests")
> -# testsuite.add_test("ACPI _PSS (Pstate) table conformance tests",
> test_pss, submenu="ACPI Tests")
> -# testsuite.add_test("ACPI _PSS (Pstate) runtime tests", test_pstates,
> submenu="ACPI Tests")
> + testsuite.add_test("ACPI _PSS (Pstate) table conformance tests",
> test_pss, submenu="ACPI Tests")
> + testsuite.add_test("ACPI _PSS (Pstate) runtime tests", test_pstates,
> submenu="ACPI Tests")
> testsuite.add_test("ACPI DSDT (Differentiated System Description
> Table)", test_dsdt, submenu="ACPI Tests")
> testsuite.add_test("ACPI FACP (Fixed ACPI Description Table)",
> test_facp, submenu="ACPI Tests")
> testsuite.add_test("ACPI HPET (High Precision Event Timer Table)",
> test_hpet, submenu="ACPI Tests")
>
> $ ./tests/venv/bin/avocado run -t acpi tests/avocado
> Fetching asset from
> tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> JOB ID : 16a8b8c7f8dc87c4b84f8a2f95135bb63464a71e
> JOB LOG :
> /home/anisinha/avocado/job-results/job-2022-10-14T22.43-16a8b8c/job.log
> (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: FAIL:
> '1' != '0'\n- 1\n+ 0\n : Some bits tests seems to have failed. Please check
> the test logs for more info. (33.01 s)
> RESULTS : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 |
> CANCEL 0
> JOB TIME : 39.17 s
>
>
> Changelog:
> v4:
> - renamed the bits test files as .py2.
> - fixed a bug with missing log file parsing. dump bits logs automatically
> for test failures (no need to re-run with V=1).
> - fixed issues with README file and enhanced it.
> - cosmetic comment updates.
> v3:
> - converted the existing test to avocado tests as per the popular
> recommendation. Added appropriate tags.
> - artifact download URL modified based on gitlab-CI URL.
>
> For biosbits repo:
> - moved to a gitlab repo with me being the maintainer.
> - added .gitlab-ci.yml file to generate the artifacts.
> v2:
> - a new class of python based tests introduced that is separate from avocado
> tests or qtests. Can be run by using "make check-pytest".
> - acpi biosbits tests are the first tests to use pytest environment.
> - bios bits tests now download the bits binary archives from a remote
> repository if they are not found locally. The test skips if download
> fails.
> - A new environment variable is introduced that can be passed by the tester
> to specify the location of the bits archives locally. test skips if the
> bits binaries are not found in that location.
> - if pip install of python module fails for whatever reaoson, the test skips.
> - misc code fixes including spell check of the README doc. README has been
> updated as well.
> - addition of SPDX license headers to bits test files.
> - update MAINTAINERS to reflect the new pytest test class.
>
> For biosbits repo:
> - added Dockerfile and build script. Made bios bits build on gcc 11.
> https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile
> https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh
> The build script generates the zip archive and tarball used by the test.
>
> v1: initial patchset. uses qtest to implement the bios bits tests.
>
> Cc: Qemu Devel <qemu-devel@nongnu.org>
> Cc: Daniel P. Berrangé <berra...@redhat.com>
> Cc: Paolo Bonzini <pbonz...@redhat.com>
> Cc: Maydell Peter <peter.mayd...@linaro.org>
> Cc: John Snow <js...@redhat.com>
> Cc: Thomas Huth <th...@redhat.com>
> Cc: Alex Bennée <alex.ben...@linaro.org>
> Cc: Igor Mammedov <imamm...@redhat.com>
> Cc: Michael Tsirkin <m...@redhat.com>
>
>
> Ani Sinha (10):
> acpi/tests/avocado/bits: initial commit of test scripts that are run
> by biosbits
> acpi/tests/avocado/bits: add SPDX license identifiers for bios bits
> tests
> acpi/tests/avocado/bits: disable acpi PSS tests that are failing in
> biosbits
> acpi/tests/avocado/bits: add smilatency test suite from bits in order
> to disable it
> acpi/tests/avocado/bits: add SPDX license identifiers for bios bits
> smilatency tests
> acpi/tests/avocado/bits: disable smilatency test since it does not
> pass everytime
> acpi/tests/avocado/bits: add biosbits config file for running bios
> tests
> acpi/tests/avocado/bits: add acpi and smbios avocado tests that uses
> biosbits
> acpi/tests/avocado/bits: add a README file to describe the test
> MAINTAINERS: add myself as the maintainer for acpi biosbits avocado
> tests
>
> MAINTAINERS | 6 +
> tests/avocado/acpi-bits.py | 353 +++
> tests/avocado/acpi-bits/README | 128 +
> .../acpi-bits/bits-config/bits-cfg.txt | 18 +
> tests/avocado/acpi-bits/bits-tests/smbios.py2 | 2434 +++++++++++++++++
> .../acpi-bits/bits-tests/smilatency.py2 | 107 +
> .../avocado/acpi-bits/bits-tests/testacpi.py2 | 287 ++
> .../acpi-bits/bits-tests/testcpuid.py2 | 87 +
> 8 files changed, 3420 insertions(+)
> create mode 100644 tests/avocado/acpi-bits.py
> create mode 100644 tests/avocado/acpi-bits/README
> create mode 100644 tests/avocado/acpi-bits/bits-config/bits-cfg.txt
> create mode 100644 tests/avocado/acpi-bits/bits-tests/smbios.py2
> create mode 100644 tests/avocado/acpi-bits/bits-tests/smilatency.py2
> create mode 100644 tests/avocado/acpi-bits/bits-tests/testacpi.py2
> create mode 100644 tests/avocado/acpi-bits/bits-tests/testcpuid.py2
--
Alex Bennée
