Re: [PATCH v3 09/24] tests/functional: enable pre-emptive caching of assets

[Philippe Mathieu-Daudé]

On 30/7/24 19:03, Daniel P. Berrangé wrote:

Many tests need to access assets stored on remote sites. We don't want
to download these during test execution when run by meson, since this
risks hitting test timeouts when data transfers are slow.

Add support for pre-emptive caching of assets by setting the env var
QEMU_TEST_PRECACHE to point to a timestamp file. When this is set,
instead of running the test, the assets will be downloaded and saved
to the cache, then the timestamp file created.

A meson custom target is created as a dependency of each test suite
to trigger the pre-emptive caching logic before the test runs.

When run in caching mode, it will locate assets by looking for class
level variables with a name prefix "ASSET_", and type "Asset".

At the ninja level

    ninja test --suite functional

will speculatively download any assets that are not already cached,
so it is advisable to set a timeout multiplier.

    QEMU_TEST_NO_DOWNLOAD=1 ninja test --suite functional

will fail the test if a required asset is not already cached

    nina check-func-precache

will download and cache all assets required by the functional
tests

At the make level, precaching is always done by

    make check-functional

Signed-off-by: Daniel P. Berrangé <berra...@redhat.com>
---
  tests/Makefile.include                 |  3 ++-
  tests/functional/meson.build           | 33 +++++++++++++++++++++++--
  tests/functional/qemu_test/asset.py    | 34 ++++++++++++++++++++++++++
  tests/functional/qemu_test/testcase.py |  8 ++++++
  4 files changed, 75 insertions(+), 3 deletions(-)

diff --git a/tests/functional/qemu_test/testcase.py 
b/tests/functional/qemu_test/testcase.py
index 27bbf4a0af..21dbd90613 100644
--- a/tests/functional/qemu_test/testcase.py
+++ b/tests/functional/qemu_test/testcase.py
@@ -21,6 +21,7 @@
  from qemu.machine import QEMUMachine
  from qemu.utils import kvm_available, tcg_available

+from .asset import Asset

  from .cmd import run_cmd
  from .config import BUILD_DIR

@@ -58,6 +59,13 @@ def tearDown(self):

          self.log.removeHandler(self._log_fh)

def main():

+        path = os.path.basename(sys.argv[0])[:-3]

This line should be added here in patch #2 in order to
avoid ...

+
+        cache = os.environ.get("QEMU_TEST_PRECACHE", None)
+        if cache is not None:
+            Asset.precache_suites(path, cache)
+            return
+
          tr = pycotap.TAPTestRunner(message_log = pycotap.LogMode.LogToError,
                                     test_output_log = 
pycotap.LogMode.LogToError)
          path = os.path.basename(sys.argv[0])[:-3]

.... this duplication.