On Mon, 27 Nov 2023, Rémi Denis-Courmont wrote:
Le maanantaina 27. marraskuuta 2023, 14.31.18 EET Martin Storsjö a écrit :
This can be useful if doing testing of uncommon CPU extensions by
running tests with QEMU (by configuring with e.g.
"target_exec=qemu-aarch64"), by only running the checkasm tests,
to get a reasonable test coverage without excessive test runtime.
For the purpose of testing future or bleeding-edge CPU extensions on
emulator, you would normally want to be able to actually filter those
in. That is more of a matter of patching checkasm than FATE.
Sorry, can you elaborate on what you mean with "filter those in" here?
Considering the poor coverage of checkasm, I fear that this just gives
the wrong impression, not to say a false sense of security. It feels
misleading to encourage or support that paradigm into FATE, in light of
that poor coverage. Afterall, if it's just about running checkasm,
anybody can just run `make tests/checkasm/checkasm &&
tests/checkasm/checkasm`.
Yes, anybody can run that - but having those results posted continuously
somewhere where other can see them can be valuable as well.
Anyway, the concrete case I'm considering, is that we've got AArch64 code
merged, that uses the I8MM extensions. We don't have any FATE
configuration that continuously test that. Whenever there are patches, I
do spin up a cloud instance that supports this extension and test the
patches there, but inbetween that we're pretty much blind.
While checkasm's coverage isn't fantastic, for this particular case I'm
not merging any AArch64 code for new extensions unless that code is
covered by checkasm.
The other AArch64 feature that we do have code for, which also is
untested, is the assembly support for branch protection and pointer
authentication. Also this is testable pretty easily with QEMU. It's of
course more interesting to run the full fate suite, but if we're not
looking for bugs in the compiler but only for bugs in our assembly, then
checkasm should cover most of it.
Yes there's potential for QEMU bugs hiding real issues, but I'd rather
have a regular run of QEMU+checkasm than not have it tested at all. And
I'm volunteering HW+time for testing these cases with QEMU for whatever
checkasm covers, but I'm not volunteering it for full fate runs with QEMU.
And sure, I can just run such configs privately, as I already do run a
bunch of various regular builds for projects I care about - but as we do
have FATE with the public status page, hooking it up to be reported there
would feel like added value for everybody.
// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
