Hello, We recently faced a problem where a DejaGnu error went un-noticed (https://gcc.gnu.org/ml/gcc-patches/2016-08/msg01879.html).
To help identify these problems earlier, here is a patch for compare_tests that will report such cases as: -------------------------------- New tests that FAIL: arm-sim: (DejaGnu) proc "scan-dump-tree-not fail_test optimized" does not exist. -------------------------------- OK? Thanks, Christophe
contrib/ChangeLog: 2016-09-20 Christophe Lyon <christophe.l...@linaro.org> * compare_tests: Take ERROR messages into account when comparing.
diff --git a/contrib/compare_tests b/contrib/compare_tests index 8ecf824..d16e7e9 100755 --- a/contrib/compare_tests +++ b/contrib/compare_tests @@ -107,8 +107,8 @@ elif [ -d "$1" -o -d "$2" ] ; then usage "Must specify either two directories or two files" fi -sed 's/^XFAIL/FAIL/; s/^XPASS/PASS/' < "$1" | awk '/^Running target / {target = $3} { if (target != "unix") { sub(/: /, "&"target": " ); }; print $0; }' | cut -c1-2000 >$tmp1 -sed 's/^XFAIL/FAIL/; s/^XPASS/PASS/' < "$2" | awk '/^Running target / {target = $3} { if (target != "unix") { sub(/: /, "&"target": " ); }; print $0; }' | cut -c1-2000 >$tmp2 +sed 's/^XFAIL/FAIL/; s/^ERROR/FAIL/; s/^XPASS/PASS/' < "$1" | awk '/^Running target / {target = $3} { if (target != "unix") { sub(/: /, "&"target": " ); }; print $0; }' | cut -c1-2000 >$tmp1 +sed 's/^XFAIL/FAIL/; s/^ERROR/FAIL/; s/^XPASS/PASS/' < "$2" | awk '/^Running target / {target = $3} { if (target != "unix") { sub(/: /, "&"target": " ); }; print $0; }' | cut -c1-2000 >$tmp2 before=$tmp1 now=$tmp2