Re: [PATCH v2 31/31] tests/functional: ignore errors when caching assets, except for 404

[Thomas Huth]

On 11/12/2024 18.26, Daniel P. Berrangé wrote:

We see periodic errors caching assets due to a combination of transient
networking and server problems. With the previous patch to skip running
a test when it has missing assets, we can now treat most cache download
errors as non-fatal.

Only HTTP 404 is retained as fatal, since it is a strong indicator of
a fully broken test rather than a transient error.

Signed-off-by: Daniel P. Berrangé <berra...@redhat.com>
---
  tests/functional/qemu_test/asset.py | 14 +++++++++++++-
  1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/tests/functional/qemu_test/asset.py 
b/tests/functional/qemu_test/asset.py
index 39832b2587..f0730695f0 100644
--- a/tests/functional/qemu_test/asset.py
+++ b/tests/functional/qemu_test/asset.py
@@ -15,6 +15,7 @@
  from time import sleep
  from pathlib import Path
  from shutil import copyfileobj
+from urllib.error import HTTPError

# Instances of this class must be declared as class level variables

@@ -170,7 +171,18 @@ def precache_test(test):
          for name, asset in vars(test.__class__).items():
              if name.startswith("ASSET_") and type(asset) == Asset:
                  log.info("Attempting to cache '%s'" % asset)
-                asset.fetch()
+                try:
+                    asset.fetch()
+                except HTTPError as e:
+                    # Treat 404 as fatal, since it is highly likely to
+                    # indicate a broken test rather than a transient
+                    # server or networking problem
+                    if e.code == 404:
+                        raise
+
+                    log.debug(f"HTTP error {e.code} from {asset.url} " +
+                              "skipping asset precache")
+
          log.removeHandler(handler)

def precache_suite(suite):

Reviewed-by: Thomas Huth <th...@redhat.com>