Le 27 novembre 2023 23:55:18 GMT+02:00, "Martin Storsjö" <mar...@martin.st> a écrit : >On Mon, 27 Nov 2023, Rémi Denis-Courmont wrote: > >> Le maanantaina 27. marraskuuta 2023, 14.31.18 EET Martin Storsjö a écrit : >>> This can be useful if doing testing of uncommon CPU extensions by >>> running tests with QEMU (by configuring with e.g. >>> "target_exec=qemu-aarch64"), by only running the checkasm tests, >>> to get a reasonable test coverage without excessive test runtime. >> >> For the purpose of testing future or bleeding-edge CPU extensions on >> emulator, you would normally want to be able to actually filter those in. >> That is more of a matter of patching checkasm than FATE. > >Sorry, can you elaborate on what you mean with "filter those in" here?
You're running all checkasm tests, not just those that require the emulator. But what's potentially much worse is that you're triggering a whole build, or it's not entirely clear from the description how you'd reuse an existing build. For Armv8, that's just bad. For RV, that's terrible, as we need to run the same checkasm with different emulator configuration (different $QEMU_CPU in the case of QEMU): one per vector length. Armv9 will potentially have the same problem if FFmpeg grows SVE(2) support. > >> Considering the poor coverage of checkasm, I fear that this just gives the >> wrong impression, not to say a false sense of security. It feels misleading >> to encourage or support that paradigm into FATE, in light of that poor >> coverage. Afterall, if it's just about running checkasm, anybody can just >> run `make tests/checkasm/checkasm && tests/checkasm/checkasm`. > >Yes, anybody can run that - but having those results posted continuously >somewhere where other can see them can be valuable as well. > >Anyway, the concrete case I'm considering, is that we've got AArch64 code >merged, that uses the I8MM extensions. We don't have any FATE configuration >that continuously test that. Whenever there are patches, I do spin up a cloud >instance that supports this extension and test the patches there, but >inbetween that we're pretty much blind. > >While checkasm's coverage isn't fantastic, for this particular case I'm not >merging any AArch64 code for new extensions unless that code is covered by >checkasm. > >The other AArch64 feature that we do have code for, which also is untested, is >the assembly support for branch protection and pointer authentication. Also >this is testable pretty easily with QEMU. It's of course more interesting to >run the full fate suite, but if we're not looking for bugs in the compiler but >only for bugs in our assembly, then checkasm should cover most of it. > >Yes there's potential for QEMU bugs hiding real issues, but I'd rather have a >regular run of QEMU+checkasm than not have it tested at all. And I'm >volunteering HW+time for testing these cases with QEMU for whatever checkasm >covers, but I'm not volunteering it for full fate runs with QEMU. > >And sure, I can just run such configs privately, as I already do run a bunch >of various regular builds for projects I care about - but as we do have FATE >with the public status page, hooking it up to be reported there would feel >like added value for everybody. > >// Martin >_______________________________________________ >ffmpeg-devel mailing list >ffmpeg-devel@ffmpeg.org >https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > >To unsubscribe, visit link above, or email >ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
