On 07/05/2025 20.45, Pierrick Bouvier wrote:
On 5/7/25 12:39 AM, Thomas Huth wrote:
Then I don't understand the previous argument from Thomas to not make
thorough the default: "The thorough functional tests download a
lot of assets from the internet, so if someone just runs "meson test"
without further parameters, I think we should not trigger these
downloads in that case". It's what precache-functional is doing.
precache-functional is *only* called when you run "make check-functional",
i.e. when you know that you want to run the functional tests that might
download assets from the internet. It's not called when you run the normal
"make check".
Are you sure ? If that's the case it was *not* my intention when i
added precaching - I thought that "make check" would call
"make check-functional" and thus trigger precaching ?
"check-functional" is not part of the normal "check" target - on purpose
since we don't want to trigger downloads by surprise when people just run
"make check". That's also why we have separate "functional" CI jobs in the
gitlab CI, since otherwise this would be handled by the normal "check" jobs
already.
`make check` calls build/pyvenv/bin/meson test --no-rebuild -t 1, which
triggers func-quick by default, triggering associated downloads, since
QEMU_TEST_NO_DOWNLOAD=1 is not set for this target, except if I missed
another hidden hack somewhere.
You missed the fact that the "quick" functional tests do not download any
assets :-)
I know it's confusing since the name of the suites rather indicate something
about the runtime of the tests and not about whether they trigger a download
or not, but the suite names are so deeply glued into the "mtest2make" logic
that I was not able to come up with a better solution. Maybe Paolo or some
other Meson expert could clean that up, but for the time being, for the
functional test we have:
- quick tests that can always run (also run during "make check")
- thorough tests that download assets from the internet (only run during
"make check-functional")
I tried to document it in docs/devel/testing/functional.rst in the "Asset
handling" section already, please have a look whether that's sufficient, or
whether you have some ideas how to improve the situation.
Thomas
