Hi - The gcc-testresults mailing list is a well-established place to plop snippets of testsuite results. It's an okay way to archive and distribute overall counts, but it's not machine readable, and it's way incomplete (lacks .log content, a lot of metadata, barely meaningfully searchable) for trying to understand why something failed.
A few years ago, our team @ RH built the bunsen system to serve as a structured repository for dejagnu (and other) testsuite types. [1] An instance running here [2] has been collecting data from the sourceware buildbots and individuals (hi iains!) for years. It convers all the toolchain pieces and then some, including gcc. It features machine readable indexing, cross-referenced test case browsing, regression analysis, old data aging, and many other capabilities. (It even has a piece that extracts data from the gcc-testresults@ mailing list archives, but there's too little there to bother. [4]) This patch attempts to make it easy for gcc developers who use the contrib/test_summary script today to opt in to contributing their results to a bunsen server (defaulting to the sourceware one [2]). The prerequites are: - git - the bunsen t-upload-git-push script [3], feel free to grab just that into your $PATH - sourceware account with bunsendb commit access [ask on admin-reque...@sourceware.org] - pretty much nothing else - specifically, gcc commit access is not necessary! Here's how it looks when it's run after a random small gcc build/test: .../contrib/test_summary -h [...] -b: instead of emailing, push test logs into a bunsen git repo -bg REPO: specify the bunsen git repo to override default -bt TAG: specify the bunsen git commit tag to override default .../contrib/test_summary -b | env PATH=$BUNSEN/INST/bin:$PATH sh -x + echo master + echo basepoints/gcc-15-3524-ga523c2ba5862 + echo a523c2ba58621c3630a1cd890d6db82879f92c90 + echo git://gcc.gnu.org/git/gcc.git + find . -name '*.log' -o -name '*.sum' -o -name '.bunsen.*' + t-upload-git-push ssh://sourceware.org/git/bunsendb.git/ fche/gcc/x86_64-20240913-1516 ec57fb8ee928e341d1f0d1b09c1d571fb590bd2b refs/tags/fche/gcc/x86_64-20240913-1516 And here's what that dataset looks like on bunsen a few minutes later: https://builder.sourceware.org/testrun/ec57fb8ee928e341d1f0d1b09c1d571fb590bd2b If this is of any interest, I'd be glad to hack on this script further to make it acceptable. [1] https://sourceware.org/bunsen/ [2] https://builder.sourceware.org/testruns/ [3] https://sourceware.org/git/?p=bunsen.git;a=blob;f=bin/t-upload-git-push [4] https://sourceware.org/git/?p=bunsen.git;a=blob;f=bin/t-sourceware-mails-import diff --git a/contrib/ChangeLog b/contrib/ChangeLog index 9b36caf02bb1..def8dd8a8a73 100644 --- a/contrib/ChangeLog +++ b/contrib/ChangeLog @@ -1,3 +1,9 @@ +2024-09-13 Frank Ch. Eigler <f...@redhat.com> + + * test_summary: Add -b (bunsen) mode to report all test results + into a https://sourceware.org/bunsen/ system instead of emailing + extracts. + 2024-08-01 Thomas Schwinge <tschwi...@baylibre.com> * gcc_update (files_and_dependencies): Update for diff --git a/contrib/test_summary b/contrib/test_summary index 5760b053ec27..867ada4d6b81 100755 --- a/contrib/test_summary +++ b/contrib/test_summary @@ -39,6 +39,9 @@ if test x"$1" = "x-h"; then should be selected from the log files. -f: force reports to be mailed; if omitted, only reports that differ from the sent.* version are sent. + -b: instead of emailing, push test logs into a bunsen git repo + -bg REPO: specify the bunsen git repo to override default + -bt TAG: specify the bunsen git commit tag to override default _EOF exit 0 fi @@ -57,6 +60,9 @@ fi : ${filesuffix=}; export filesuffix : ${move=true}; export move : ${forcemail=false}; export forcemail +: ${bunsen=false}; +: ${bunsengit=ssh://sourceware.org/git/bunsendb.git/}; +: ${bunsentag=`whoami`/gcc/`uname -m`-`date +%Y%m%d-%H%M`}; while true; do case "$1" in -o) filesuffix=.sent; move=false; : ${mailto=nobody}; shift;; @@ -64,10 +70,30 @@ while true; do -p) prepend_logs=${prepend_logs+"$prepend_logs "}"$2"; shift 2;; -i) append_logs=${append_logs+"$append_logs "}"$2"; shift 2;; -m) mailto=$2; forcemail=true; shift 2;; + -b) bunsen=true; shift;; + -bg) bunsengit=$2; shift 2;; + -bt) bunsentag=$2; shift 2;; -f) unset mailto; forcemail=true; shift;; *) break;; esac done +if [ "x$bunsen" = "xtrue" ]; then + gitsrcdir=`dirname "$0"` # this script, contrib/test_summary + gitsrcdir=`dirname "$gitsrcdir"` # and the parent directory + if [ -d "$gitsrcdir/.git" ]; then # is this a git-hosted source tree? + # gather basic build metadata for sourceware-buildbot-style .bunsen data + gitbranch=`cd "$gitsrcdir"; git rev-parse --abbrev-ref HEAD` + echo "echo '$gitbranch' > .bunsen.source.gitbranch &&" + gitdescribe=`cd "$gitsrcdir"; git describe` + echo "echo '$gitdescribe' > .bunsen.source.gitdescribe &&" + gitname=`cd "$gitsrcdir"; git rev-parse HEAD` + echo "echo '$gitname' > .bunsen.source.gitname &&" + gitremote=`cd "$gitsrcdir"; git config --get remote.origin.url` + echo "echo '$gitremote' > .bunsen.source.gitrepo &&" + fi + echo "(find . -name '*.log' -o -name '*.sum' -o -name '.bunsen.*' | t-upload-git-push '$bunsengit' '$bunsentag')" + exit 0 +fi : ${mailto="\" address \""}; export mailto files=`find . -name \*.sum$filesuffix -print | sort` anyfile=false anychange=$forcemail &&
