On Tue, 21 May 2024, Rémi Denis-Courmont wrote:
Le 21 mai 2024 09:37:18 GMT+03:00, "Martin Storsjö" <mar...@martin.st> a écrit :
On Tue, 21 May 2024, Rémi Denis-Courmont wrote:
Hi,
VVC benchmarks have increased checksam runtime by at least an order of
magnitude. It's become so prohibitively slow that I could not even get
to the end.
This is not an acceptable situation and impedes non-VVC assembler work
I don't quite understand; whenever benchmarking anything in checkasm, I
would always run e.g. "checkasm --test=ac3dsp
--bench=ac3_sum_square_bufferfly_float", limiting the total running of
tests to a specific module, and only benchmarking a subset of the run
functions. (The --bench parameter specifies a prefix; only functions
matching that prefix gets benchmarked.)
Sure that's how you do it when you're working on a specific new
optimisation. Now we're trying to compare 128-bit and 256-bit vectors
for *all* existing functions to see which ones need to be reworked.
That used to work (in 30 minutes on K230, 5 minutes on Zen 2, IIRC). Now
it's effectively broken and that's not acceptable'
Ah, I see. Ok, that's a reasonable thing to do I guess.
(It's of course possible to speed it up further by only testing specific
--test=foo cases where you know you have riscv assembly worth
benchmarking, but if it was doable in a tolerable amount of time before,
that shouldn't be needed.)
Without limiting the scope with a --test parameter, checkasm
benchmarking has always been prohibitively slow for me - so I don't
think there's anything new here?
As said, it seems to be literally an order of magnitude slower than
before if not worse.
That said I'm not familiar with the VVC tests in checkasm, perhaps they
benchmark things excessively. But I don't see how that would impede
work on other DSP functions in any way?
James also complained about the same thing before I.
Indeed, the tests in vvc_alf group seem to do excessive benchmarking
(benchmarking every width/height combination between 4 and 128, in
increments of 4). I sent a patch to cut this down to a reasonable amount.
Overall, I would expect the vvc checkasm tests to take a notable amount of
time. Dav1d's checkasm takes twice as long to run as ffmpeg's, and it's
probably a reasonable to assume that vvc is roughly of the same level of
complexity as av1, so it's probably expected that ffmpeg's checkasm
runtime at least doubles, once all vvc routines are integrated in
checkasm.
But the tests in vvc_alf indeed had an entirely unreasonable amount of
benchmarking hooked up, and that should indeed be fixed, e.g. with the
patch I just sent.
// Martin
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
