Hi Martin!
On 2024-11-07 11:36 +0200, Martin Storsjö wrote:
> If running tests with "make -j<N> fate", the execution will stop
> after the first failing test. To get an overview of the whole
> test suite, one rather would run "make -k -j<N> fate", which then
> again buries the results about what tests actually failed further
> up in the console log.
>
> Add a target so one can run "make fate-list-failing", to see a list
> of all tests that failed the last time they were executed.
>
> Also add a companion target "fate-clear-results" which removes all
> the old test results. (When executing a subset of tests, the result
> files of all tests that aren't executed stay untouched. This also
> allows getting rid of results for tests that no longer are present
> in the testsuite.)
Looks good and useful to me.
Just a few ideas follow:
> ---
> So far, I've always just manually run
> "cat tests/data/fate/*.rep | cut -f 1-2 -d : | grep -v :0", but
> perhaps we should at least wrap it up in something more convenient
> for the other developers.
> ---
> doc/build_system.txt | 6 ++++++
> doc/fate.texi | 7 +++++++
> tests/Makefile | 6 ++++++
> 3 files changed, 19 insertions(+)
>
> diff --git a/doc/build_system.txt b/doc/build_system.txt
> index 0b1b0c2054..91c7a5e9af 100644
> --- a/doc/build_system.txt
> +++ b/doc/build_system.txt
> @@ -30,6 +30,12 @@ fate
> fate-list
> List all fate/regression test targets.
>
> +fate-list-failing
> + List the fate tests that failed the last time they were executed.
> +
> +fate-clear-results
> + Remove the test results from previous test executions.
> +
Would it be better to use the same description as int fate.texi ?
> install
> Install headers, libraries and programs.
>
> diff --git a/doc/fate.texi b/doc/fate.texi
> index 17644ce65a..bf01816af8 100644
> --- a/doc/fate.texi
> +++ b/doc/fate.texi
> @@ -208,6 +208,13 @@ Download/synchronize sample files to the configured
> samples directory.
> @item fate-list
> Will list all fate/regression test targets.
>
> +@item fate-list-failing
> +List the fate tests that failed the last time they were executed.
> +
> +@item fate-clear-results
> +Remove the test results from previous test executions (getting rid of
> +potentially stale results from fate-list-failing).
> +
> @item fate
> Run the FATE test suite (requires the fate-suite dataset).
> @end table
> diff --git a/tests/Makefile b/tests/Makefile
> index 9b70145015..adb0799328 100644
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -313,6 +313,12 @@ $(FATE): $(FATE_UTILS:%=tests/%$(HOSTEXESUF)) |
> $(FATE_OUTDIRS)
> fate-list:
> @printf '%s\n' $(sort $(FATE))
>
> +fate-list-failing:
> + @cat tests/data/fate/*.rep | cut -f 1-2 -d : | grep -v :0 | sed
> 's/:.*//;s/^/fate-/'
> +
> +fate-clear-results:
> + @rm -f tests/data/fate/*.rep
> +
> coverage.info: TAG = LCOV
> coverage.info:
> $(M)lcov -q -d $(CURDIR) -b $(patsubst src%,./,$(SRC_LINK)) --capture |
> \
> --
Maybe the attached patch would make sense on top of your changes?
Best regards,
Alexander
From 130ca8d85dcb1fecace596e1f23e0414d56348ef Mon Sep 17 00:00:00 2001
From: Alexander Strasser <eclip...@gmx.net>
Date: Sun, 1 Dec 2024 15:06:53 +0100
Subject: [PATCH] fate: Simplify implementation for target fate-list-failing
Make it a little easier to read and use one awk invocation as opposed
to a combination of cat, cut, grep, and sed.
Signed-off-by: Alexander Strasser <eclip...@gmx.net>
---
tests/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/Makefile b/tests/Makefile
index adb0799328..2c4ad7c18f 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -314,7 +314,7 @@ fate-list:
@printf '%s\n' $(sort $(FATE))
fate-list-failing:
- @cat tests/data/fate/*.rep | cut -f 1-2 -d : | grep -v :0 | sed
's/:.*//;s/^/fate-/'
+ @awk -F: '$$2 != 0 { print "fate-" $$1 }' tests/data/fate/*.rep
fate-clear-results:
@rm -f tests/data/fate/*.rep
--
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
