On Wed, Aug 21, 2024 at 10:27:36AM +0200, Thomas Huth wrote:
> Document the new functional testing framework. The text is originally
> based on the Avocado documentation, but heavily modified to match the
> new framework.
>
> Signed-off-by: Thomas Huth <th...@redhat.com>
> ---
> docs/devel/testing/functional.rst | 269 ++++++++++++++++++++++++++++++
> docs/devel/testing/index.rst | 1 +
> docs/devel/testing/main.rst | 12 ++
> 3 files changed, 282 insertions(+)
> create mode 100644 docs/devel/testing/functional.rst
Reviewed-by: Daniel P. Berrangé <berra...@redhat.com>
> +The tests should be written in the style of the Python `unittest`_
> +framework, using stdio for the TAP protocol. The folder
> +``tests/functional/qemu_test`` provides classes (e.g. the ``QemuBaseTest``
> +and the ``QemuSystemTest`` classes) and utility functions that help
> +to get your test into the right shape.
One gotcha when using TAP protocol is that you can't just spew
debug info to stdout/stderr. Each line of debug info needs to
be prefixed with '#' so it is interpreted as diagnostic output.
We should point this out and recommend people to exclusively
using the 'logging' framework.
Particular care should be taken when spawning sub-processes
to capture both stdout/stderr and then log the result if
needed.
> +Overview
> +--------
> +
> +The ``tests/functional/qemu_test`` directory provides the ``qemu_test``
> +Python module, containing the ``qemu_test.QemuSystemTest`` class.
> +Here is a simple usage example:
> +
> +.. code::
> +
> + #!/usr/bin/env python3
> +
> + from qemu_test import QemuSystemTest
> +
> + class Version(QemuSystemTest):
> +
> + def test_qmp_human_info_version(self):
> + self.vm.launch()
> + res = self.vm.cmd('human-monitor-command',
> + command_line='info version')
> + self.assertRegex(res, r'^(\d+\.\d+\.\d)')
> +
> + if __name__ == '__main__':
> + QemuSystemTest.main()
> +
> +By providing the "hash bang" line at the beginning of the script,
> +and by calling into QemuSystemTest.main() when it is run directly,
> +the test framework makes sure to run all test_*() functions in the
> +right fassion (e.g. with TAP output that is required by the meson test
> +runner).
Perhaps say that the test file should have execute permissions,
given the hash bang ?
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
