On Tue, Mar 31, 2020 at 5:07 PM Philippe Mathieu-Daudé <phi...@redhat.com> wrote: <snip> > > First job failed by timeout, 2nd succeeded: > https://travis-ci.org/github/philmd/qemu/jobs/669265466 > > However "Ran for 46 min 48 sec" > > From the log: > > Fetching asset from > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_mips64el_malta_5KEc_cpio > Fetching asset from > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_mips64el_malta_5KEc_cpio > Fetching asset from > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi > Fetching asset from > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd > Fetching asset from > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd > Fetching asset from > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd > Fetching asset from > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd > Fetching asset from > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic > Fetching asset from > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9 > Fetching asset from > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9 > Fetching asset from > tests/acceptance/ppc_prep_40p.py:IbmPrep40pMachine.test_openbios_and_netbsd > ... > (13/82) > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_mips64el_malta_5KEc_cpio: > SKIP: untrusted code > (24/82) > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic: > SKIP: storage limited > ... > (25/82) > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9: > SKIP: storage limited > ... > (63/82) > tests/acceptance/ppc_prep_40p.py:IbmPrep40pMachine.test_openbios_and_netbsd: > SKIP: Running on Travis-CI > > Is it possible that we are now fetching assets for tests we are not > running? In particular the one marked @skip because the downloading time > was too long on Travis?
Yes, your assumption is correct, this execution of Avocado downloaded assets for tests that were skipped. Let me try to explain how the asset feature works today on Avocado. Avocado has two basic ways to work with assets: 1. Parse limited use cases of `fetch_asset` call in the test file and execute them. This operation can happen in two different scenarios. First, when using the command line `avocado assets fetch <test_file>`. In this case, it is a standalone execution of each fetch call and the test is not executed at all. Second, by running the test. The, enabled by default, plugin FetchAssetJob will do the same operation of parsing the test file and executing occurrences of `fetch_asset` call before the tests start to run. Again, the fetch time is not computed in the job time. 2. Execute the `fetch_asset` call from each test during the test execution. In this case, the FetchAssetJob plugin should be disabled. The fetch time is added to the job time as the asset download occurs during the test execution. The acceptance tests which make use of `fetch_asset` are all using the first method with FetchAssetJob plugin enabled. As Avocado is parsing the test file before it starts to run the tests, it is not aware of possible skips that may occur during a test execution due to possible dynamic dependency. This is not the desired behavior, as you mentioned, Avocado is downloading an asset that will not be used because its test will be skipped. To minimize the damage on the download side, the Travis job is holding the avocado cache. It means the download should happen just once. This does not minimize the damage to space usage. One possible workaround here is to temporarily disable the FetchAssetJob plugin, now that the needed assets are on Travis Avocado cache. The downside is that when an asset is not available in the cache, it will be downloaded during the test execution and the download time will be added to the job time. I don't know if it is possible to manually remove an asset from Travis Avocado cache. If so, this can be done for the tests that should be skipped because of space usage. We have been trying to make the Asset feature as flexible as possible to accommodate the use cases we have been identifying. Thanks for reporting this! > > RESULTS : PASS 65 | ERROR 0 | FAIL 0 | SKIP 14 | WARN 0 | INTERRUPT 0 > | CANCEL 3 > JOB TIME : 1480.72 s > > Does this "JOB TIME" sums the 'Fetching asset' part? Answered in the comments above. > > Thanks, > > Phil. > Willian
