Ani Sinha <a...@anisinha.ca> writes:
> Please see the README file in patch 9 for more details. > Sample runs are as follows: > > Passed tests: > > $ ./tests/venv/bin/avocado run -t acpi tests/avocado > Fetching asset from > tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits > JOB ID : e95c7413e996bfb59389839e5ca5105464ef098f > JOB LOG : > /home/anisinha/avocado/job-results/job-2022-10-14T19.15-e95c741/job.log > (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS > (33.08 s) > RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | > CANCEL 0 > JOB TIME : 39.18 s Hmm something went wrong: ➜ ./tests/venv/bin/avocado run -t acpi tests/avocado Fetching asset from tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits JOB ID : 08b6d26bf52a22ccbe1908b14d5b7b27f76ad7fa JOB LOG : /home/alex/avocado/job-results/job-2022-10-18T16.29-08b6d26/job.log (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: ERROR: Error while generating the bits iso. Pass V=1 in the environment to get more details. Command '[ '/var/tmp/acpi-bits-yy4mfxrz.tmp/grub-inst-x86_64-efi/bin/grub-mkrescue', '-o', '/var/tmp/acpi-bits-yy4mfxrz.tmp/bits-2020.iso', '/var/tmp/acpi-bits-yy4mfxrz.t... (7.21 s) RESULTS : PASS 0 | ERROR 1 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 7.79 s have we added another binary dependency? > > > $ ./tests/venv/bin/avocado run ./tests/avocado/acpi-bits.py > Fetching asset from > ./tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits > JOB ID : c6e588594bceb2c5fdbdf45095559c58e4030f9c > JOB LOG : > /home/anisinha/avocado/job-results/job-2022-10-14T19.13-c6e5885/job.log > (1/1) ./tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS > (32.29 s) > RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | > CANCEL 0 > JOB TIME : 39.27 s > > When PSS tests are enabled: > > $ git diff > diff --git a/tests/avocado/acpi-bits/bits-tests/testacpi.py2 > b/tests/avocado/acpi-bits/bits-tests/testacpi.py2 > index f818a9cce6..18dc818d62 100644 > --- a/tests/avocado/acpi-bits/bits-tests/testacpi.py2 > +++ b/tests/avocado/acpi-bits/bits-tests/testacpi.py2 > @@ -40,8 +40,8 @@ import time > > def register_tests(): > testsuite.add_test("ACPI _MAT (Multiple APIC Table Entry) under > Processor objects", test_mat, submenu="ACPI Tests") > -# testsuite.add_test("ACPI _PSS (Pstate) table conformance tests", > test_pss, submenu="ACPI Tests") > -# testsuite.add_test("ACPI _PSS (Pstate) runtime tests", test_pstates, > submenu="ACPI Tests") > + testsuite.add_test("ACPI _PSS (Pstate) table conformance tests", > test_pss, submenu="ACPI Tests") > + testsuite.add_test("ACPI _PSS (Pstate) runtime tests", test_pstates, > submenu="ACPI Tests") > testsuite.add_test("ACPI DSDT (Differentiated System Description > Table)", test_dsdt, submenu="ACPI Tests") > testsuite.add_test("ACPI FACP (Fixed ACPI Description Table)", > test_facp, submenu="ACPI Tests") > testsuite.add_test("ACPI HPET (High Precision Event Timer Table)", > test_hpet, submenu="ACPI Tests") > > $ ./tests/venv/bin/avocado run -t acpi tests/avocado > Fetching asset from > tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits > JOB ID : 16a8b8c7f8dc87c4b84f8a2f95135bb63464a71e > JOB LOG : > /home/anisinha/avocado/job-results/job-2022-10-14T22.43-16a8b8c/job.log > (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: FAIL: > '1' != '0'\n- 1\n+ 0\n : Some bits tests seems to have failed. Please check > the test logs for more info. (33.01 s) > RESULTS : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | > CANCEL 0 > JOB TIME : 39.17 s > > > Changelog: > v4: > - renamed the bits test files as .py2. > - fixed a bug with missing log file parsing. dump bits logs automatically > for test failures (no need to re-run with V=1). > - fixed issues with README file and enhanced it. > - cosmetic comment updates. > v3: > - converted the existing test to avocado tests as per the popular > recommendation. Added appropriate tags. > - artifact download URL modified based on gitlab-CI URL. > > For biosbits repo: > - moved to a gitlab repo with me being the maintainer. > - added .gitlab-ci.yml file to generate the artifacts. > v2: > - a new class of python based tests introduced that is separate from avocado > tests or qtests. Can be run by using "make check-pytest". > - acpi biosbits tests are the first tests to use pytest environment. > - bios bits tests now download the bits binary archives from a remote > repository if they are not found locally. The test skips if download > fails. > - A new environment variable is introduced that can be passed by the tester > to specify the location of the bits archives locally. test skips if the > bits binaries are not found in that location. > - if pip install of python module fails for whatever reaoson, the test skips. > - misc code fixes including spell check of the README doc. README has been > updated as well. > - addition of SPDX license headers to bits test files. > - update MAINTAINERS to reflect the new pytest test class. > > For biosbits repo: > - added Dockerfile and build script. Made bios bits build on gcc 11. > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/Dockerfile > https://github.com/ani-sinha/bits/blob/bits-qemu-logging/build-artifacts.sh > The build script generates the zip archive and tarball used by the test. > > v1: initial patchset. uses qtest to implement the bios bits tests. > > Cc: Qemu Devel <qemu-devel@nongnu.org> > Cc: Daniel P. Berrangé <berra...@redhat.com> > Cc: Paolo Bonzini <pbonz...@redhat.com> > Cc: Maydell Peter <peter.mayd...@linaro.org> > Cc: John Snow <js...@redhat.com> > Cc: Thomas Huth <th...@redhat.com> > Cc: Alex Bennée <alex.ben...@linaro.org> > Cc: Igor Mammedov <imamm...@redhat.com> > Cc: Michael Tsirkin <m...@redhat.com> > > > Ani Sinha (10): > acpi/tests/avocado/bits: initial commit of test scripts that are run > by biosbits > acpi/tests/avocado/bits: add SPDX license identifiers for bios bits > tests > acpi/tests/avocado/bits: disable acpi PSS tests that are failing in > biosbits > acpi/tests/avocado/bits: add smilatency test suite from bits in order > to disable it > acpi/tests/avocado/bits: add SPDX license identifiers for bios bits > smilatency tests > acpi/tests/avocado/bits: disable smilatency test since it does not > pass everytime > acpi/tests/avocado/bits: add biosbits config file for running bios > tests > acpi/tests/avocado/bits: add acpi and smbios avocado tests that uses > biosbits > acpi/tests/avocado/bits: add a README file to describe the test > MAINTAINERS: add myself as the maintainer for acpi biosbits avocado > tests > > MAINTAINERS | 6 + > tests/avocado/acpi-bits.py | 353 +++ > tests/avocado/acpi-bits/README | 128 + > .../acpi-bits/bits-config/bits-cfg.txt | 18 + > tests/avocado/acpi-bits/bits-tests/smbios.py2 | 2434 +++++++++++++++++ > .../acpi-bits/bits-tests/smilatency.py2 | 107 + > .../avocado/acpi-bits/bits-tests/testacpi.py2 | 287 ++ > .../acpi-bits/bits-tests/testcpuid.py2 | 87 + > 8 files changed, 3420 insertions(+) > create mode 100644 tests/avocado/acpi-bits.py > create mode 100644 tests/avocado/acpi-bits/README > create mode 100644 tests/avocado/acpi-bits/bits-config/bits-cfg.txt > create mode 100644 tests/avocado/acpi-bits/bits-tests/smbios.py2 > create mode 100644 tests/avocado/acpi-bits/bits-tests/smilatency.py2 > create mode 100644 tests/avocado/acpi-bits/bits-tests/testacpi.py2 > create mode 100644 tests/avocado/acpi-bits/bits-tests/testcpuid.py2 -- Alex Bennée