Background ========== A few months ago, Cleber started a thread[1] about the system for Avocado-based tests. He invited people to take a look at it, we got some interesting feedback.
However, I saw no feedback on the actual code itself, and I don't know what needs to be done to get this merged. To make sure we're having a discussion about the implementation of avocado_qemu and to figure out what's still necessary to get it included, I am sending all the commits from Amador's avocado_qemu branch[2] as a RFC. Note that I didn't review most of this code yet, and I will probably send comments and questions as replies to this RFC later. Also, many of the patches in this series probably can be squashed together, but I decided to not do that before getting some feedback on the changes. [1] https://www.mail-archive.com/qemu-devel@nongnu.org/msg506859.html [2] https://github.com/apahim/qemu/commits/avocado_qemu README ====== Below, I'm copying the raw contents of tests/avocado/README.rst, for reference: ---------------------------------------------------------------------- ======================================== QEMU tests using the Avocado Framework ======================================== This directory [tests/avocado] hosts functional tests written using Avocado Testing Framework. Installation ============ To install Avocado and the dependencies needed for these tests, run:: pip install --user avocado-framework avocado-framework-plugin-varianter-yaml-to-mux aexpect Alternatively, follow the instructions on this link:: http://avocado-framework.readthedocs.io/en/latest/GetStartedGuide.html#installing-avocado Overview ======== In this directory [tests/avocado], an ``avocado_qemu`` package is provided, containing the ``test`` module, which inherits from ``avocado.Test`` and provides a builtin and easy-to-use Qemu virtual machine. Here's a template that can be used as reference to start writing your own tests:: from avocado_qemu import test class MyTest(test.QemuTest): """ :avocado: enable """ def setUp(self): self.vm.args.extend(['-m', '512']) self.vm.launch() def test_01(self): res = self.vm.qmp('human-monitor-command', command_line='info version') self.assertIn('v2.9.0', res['return']) def tearDown(self): self.vm.shutdown() To execute your test, run:: avocado run test_my_test.py To execute all tests, run:: avocado run . If you don't specify the Qemu binary to use, the ``avocado_qemu`` package will automatically probe it. The probe will try to use the Qemu binary from the git tree build directory, using the same architecture as the local system (if the architecture is not specified). If the Qemu binary is not available in the git tree build directory, the next try is to use the system installed Qemu binary. You can define a number of optional parameters, providing them via YAML file using the Avocado parameters system: - ``qemu_bin``: Use a given Qemu binary, skipping the automatic probe. Example: ``qemu_bin: /usr/libexec/qemu-kvm``. - ``qemu_dst_bin``: Use a given Qemu binary to create the destination VM when the migration process takes place. If it's not provided, the same binary used in the source VM will be used for the destination VM. Example: ``qemu_dst_bin: /usr/libexec/qemu-kvm-binary2``. - ``arch``: Probe the Qemu binary from a given architecture. It has no effect if ``qemu_bin`` is specified. If not provided, the binary probe will use the system architecture. Example: ``arch: x86_64`` - ``image_path``: When a test requires (usually a bootable) image, this parameter is used to define where the image is located. When undefined it uses ``$QEMU_ROOT/bootable_image_$arch.qcow2``. The image is added to the qemu command __only__ when the test requires an image. By default ``,snapshot=on`` is used, but it can be altered by ``image_snapshot`` parameter. - ``image_user`` and ``image_pass``: When using a ``image_path``, if you want to get the console from the Guest OS you have to define the Guest OS credentials. Example: ``image_user: avocado`` and ``image_pass: p4ssw0rd``. Both parameters have defaults to ``avocado``. - ``machine_type``: Use this option to define a machine type for the VM. Example: ``machine_type: pc`` - ``machine_accel``: Use this option to define a machine acceleration for the VM. Example: ``machine_accel: kvm``. - ``machine_kvm_type``: Use this option to select the KVM type when the ``accel`` is ``kvm`` and there are more than one KVM types available. Example: ``machine_kvm_type: PR`` Run the test with:: $ avocado run test_my_test.py -m parameters.yaml Additionally, you can use a variants file to to set different values for each parameter. Using the YAML tag ``!mux`` Avocado will execute the tests once per combination of parameters. Example:: $ cat variants.yaml architecture: !mux x86_64: arch: x86_64 i386: arch: i386 Run it the with:: $ avocado run test_my_test.py -m variants.yaml You can use both the parameters file and the variants file in the same command line:: $ avocado run test_my_test.py -m parameters.yaml variants.yaml Avocado will then merge the parameters from both files and create the proper variants. See ``avocado run --help`` and ``man avocado`` for several other options, such as ``--filter-by-tags``, ``--show-job-log``, ``--failfast``, etc. Uninstallation ============== If you've followed the installation instructions above, you can easily uninstall Avocado. Start by listing the packages you have installed:: pip list --user And remove any package you want with:: pip uninstall <package_name> Amador Pahim (12): qemu.py: Introduce _create_console() method Introduce the basic framework to run Avocado tests avocado_qemu: Fix exception name in caller avocado_qemu: Improve migration error message avocado_qemu: Functional test for RHBZ#1431939 avocado_qemu: Functional test for RHBZ#1447027 avocado_qemu: Functional test for RHBZ#1436616 avocado_qemu: Clean unneeded 'pass' avocado_qemu: Set QMP log level to INFO avocado_qemu: Introduce the add_image() VM API avocado_qemu: Tests fixes avocado_qemu: Force vmimage distro Cleber Rosa (3): avocado_qemu: Remove duplicate PortTracker implementation avocado_qemu: Simplify the installation instructions avocado_qemu: Add a few VNC related tests Lukáš Doktor (9): avocado_qemu: Improve handle_prompts to allow login after booted vm avocado_qemu: Be lenient towards poluted serial console avocado_qemu: Increase the login timeout to 60s avocado_qemu: Add " " after the default prompt regexp avocado_qemu: Store "arch" in VM avocado_qemu: Provide defaults for user and pass avocado_qemu: Ignore kernel messages on get_console avocado_qemu: Add support to request image for testing avocado_qemu: Functional test for RHBZ1473203 scripts/qemu.py | 59 ++- tests/avocado/README.rst | 132 +++++++ tests/avocado/avocado_qemu/__init__.py | 0 tests/avocado/avocado_qemu/test.py | 418 +++++++++++++++++++++ tests/avocado/parameters.yaml | 19 + tests/avocado/test_info_memdev_host_nodes.py | 66 ++++ tests/avocado/test_nec-usb-xhci.py | 63 ++++ .../test_nec-usb-xhci.py.data/parameters.yaml | 4 + tests/avocado/test_numa_hotplug.py | 120 ++++++ tests/avocado/test_ovmf_with_240_vcpus.py | 70 ++++ .../parameters.yaml | 2 + tests/avocado/test_vnc.py | 58 +++ tests/avocado/variants.yaml | 62 +++ tests/qemu-iotests/iotests.py | 28 +- 14 files changed, 1077 insertions(+), 24 deletions(-) create mode 100644 tests/avocado/README.rst create mode 100644 tests/avocado/avocado_qemu/__init__.py create mode 100644 tests/avocado/avocado_qemu/test.py create mode 100644 tests/avocado/parameters.yaml create mode 100644 tests/avocado/test_info_memdev_host_nodes.py create mode 100644 tests/avocado/test_nec-usb-xhci.py create mode 100644 tests/avocado/test_nec-usb-xhci.py.data/parameters.yaml create mode 100644 tests/avocado/test_numa_hotplug.py create mode 100644 tests/avocado/test_ovmf_with_240_vcpus.py create mode 100644 tests/avocado/test_ovmf_with_240_vcpus.py.data/parameters.yaml create mode 100644 tests/avocado/test_vnc.py create mode 100644 tests/avocado/variants.yaml -- 2.14.3
