On 25/07/2025 16.48, Daniel P. Berrangé wrote:
On Fri, Jul 25, 2025 at 02:25:46PM +0100, Daniel P. Berrangé wrote:
On Fri, Jul 25, 2025 at 12:41:25PM +0300, Manos Pitsidianakis wrote:
Add a CLI argument that takes fnmatch(3)-style patterns as value and can
be specified many times. Only tests that match the pattern will be
executed. This argument is passed to unittest.main which takes the same
argument.
Acked-by: Thomas Huth <th...@redhat.com>
Signed-off-by: Manos Pitsidianakis <manos.pitsidiana...@linaro.org>
---
tests/functional/qemu_test/testcase.py | 23 +++++++++++++++++++++--
1 file changed, 21 insertions(+), 2 deletions(-)
One of the goals with the new functional test system was that we stop trying
to (re-)invent a custom test runner harness, as was the case with Avocado,
in favour of relying on the pre-existing python infrastructure to the
greatest extent possible.
Seeing this, and all the other CLI arg handling added in this series, makes
me fairly uncomfortable, as it is effectively inventing a custom test runner
once again which is exactly what we wanted to get away from.
At the same time, there are some pieces in this series that do things that
unittest.main() can't do on its own.
So considering the broader picture, we already have a load of tunables
on the test execution that we control exclusively via envirnoment
variables, as that gives us independance of the test runner, which
owns sys.argv processing.
So in terms of this series, IMHO, we should just add support for
QEMU_TEST_DEBUG=1 and QEMU_TEST_KEEP_SCRATCH=1 as two new tunables,
and not touch sys.argv at all.
I basically agree, but I wonder if we could maybe also have both? First
parse the sys.argv for our parameters, then pass the unknown ones to
unittest.main() ?
With that, the only thing we're missing is a way to enumerate the
test cases, but IMHO that's the job of the test runner and thus
out of scope for QEMU to solve.
With the patch applied that passes the argv to unittest.main(), there also
seems to be a "discover" mode:
~/devel/qemu/tests/functional/test_vnc.py -h | grep usage:
usage: test_vnc.py [-h] [-v] [-q] [--locals] [--durations N] [-f] [-c] [-b]
usage: test_vnc.py discover [-h] [-v] [-q] [--locals] [--durations N] [-f]
... however, when I try to run it, it does not work as expected (I get a
"ModuleNotFoundError: No module named 'discover'" error).
Thomas
