On 27 September 2023 16:47:27 CEST, Maxim Kuvyrkov <maxim.kuvyr...@linaro.org> wrote: >Hi Bernhard, > >Thanks, I meant to fix this, but forgot.
np. >The underlying problem here is that we want to detect which sub-testsuites had >failures. Current regex doesn't match go's case because there is no "..." at >the end: "Running foo" vs "Running foo ..." . > >My preferred way of fixing this is to make go's testsuite print out "..." . >We have a similar patch for glibc [1]. > >[1] https://sourceware.org/pipermail/libc-alpha/2023-June/148702.html Which asks: ---8<--- >> WDYT? > > I looked at the gcc-testresults mailing list, and there appear no > === … failures === lines at all? What was the motivation for adding it > in the first place? The only motivation is that it looks like a nice header for the following FAILs. What's your preference for the line -- drop it entirely or print out: === glibc failures === no unexpected failures ? ---8<--- I'd drop the above entirely if there are no failures, it's pretty superfluous, isn't it. And concerning gotools and the missing trailing ".exp ...", I guess it's fine to add that to streamline the gotools output to all the other existing sum output. TIA, > >-- >Maxim Kuvyrkov >https://www.linaro.org > >> On Sep 26, 2023, at 19:46, Bernhard Reutner-Fischer <rep.dot....@gmail.com> >> wrote: >> >> Hi Maxim! >> >> On Mon, 5 Jun 2023 18:06:25 +0400 >> Maxim Kuvyrkov via Gcc-patches <gcc-patches@gcc.gnu.org> wrote: >> >>>> On Jun 3, 2023, at 19:17, Jeff Law <jeffreya...@gmail.com> wrote: >>>> >>>> On 6/2/23 09:20, Maxim Kuvyrkov via Gcc-patches wrote: >>>>> This patch adds tracking of current testsuite "tool" and "exp" >>>>> to the processing of .sum files. This avoids aliasing between >>>>> tests from different testsuites with same name+description. >>>>> E.g., this is necessary for testsuite/c-c++-common, which is ran >>>>> for both gcc and g++ "tools". >>>>> This patch changes manifest format from ... >>>>> <cut> >>>>> FAIL: gcc_test >>>>> FAIL: g++_test >>>>> </cut> >>>>> ... to ... >>>>> <cut> >>>>> === gcc tests === >>>>> Running gcc/foo.exp ... >>>>> FAIL: gcc_test >>>>> === gcc Summary == >>>>> === g++ tests === >>>>> Running g++/bar.exp ... >>>>> FAIL: g++_test >>>>> === g++ Summary == >>>>> </cut>. >>>>> The new format uses same formatting as DejaGnu's .sum files >>>>> to specify which "tool" and "exp" the test belongs to. >>>> I think the series is fine. You're not likely to hear from Diego or Doug >>>> I suspect, I don't think either are involved in GNU stuff anymore. >>>> >>> >>> Thanks, Jeff. I'll wait for a couple of days and will merge if there are >>> no new comments. >> >> Maxim, may i ask you to have a look at the following problem, please? >> >> ISTM that your exp code does not work as expected for go, maybe you >> forgot to test the changes with go enabled? >> >> Ever since your changes in summer i see the following: >> >> gcc-14.mine$ >> /scratch/src/gcc-14.mine/contrib/testsuite-management/validate_failures.py >> --clean_build ../gcc-14.orig/ >> Getting actual results from build directory . >> ./gcc/testsuite/go/go.sum >> ./gcc/testsuite/gcc/gcc.sum >> ./gcc/testsuite/objc/objc.sum >> ./gcc/testsuite/jit/jit.sum >> ./gcc/testsuite/gdc/gdc.sum >> ./gcc/testsuite/gnat/gnat.sum >> ./gcc/testsuite/ada/acats/acats.sum >> ./gcc/testsuite/g++/g++.sum >> ./gcc/testsuite/obj-c++/obj-c++.sum >> ./gcc/testsuite/rust/rust.sum >> ./gcc/testsuite/gfortran/gfortran.sum >> ./x86_64-pc-linux-gnu/libgomp/testsuite/libgomp.sum >> ./x86_64-pc-linux-gnu/libphobos/testsuite/libphobos.sum >> ./x86_64-pc-linux-gnu/libstdc++-v3/testsuite/libstdc++.sum >> ./x86_64-pc-linux-gnu/libffi/testsuite/libffi.sum >> ./x86_64-pc-linux-gnu/libitm/testsuite/libitm.sum >> ./x86_64-pc-linux-gnu/libgo/libgo.sum >> ./x86_64-pc-linux-gnu/libatomic/testsuite/libatomic.sum >> ./gotools/gotools.sum >> .sum file seems to be broken: tool="gotools", exp="None", >> summary_line="FAIL: TestScript" >> Traceback (most recent call last): >> File >> "/scratch/src/gcc-14.mine/contrib/testsuite-management/validate_failures.py", >> line 732, in <module> >> retval = Main(sys.argv) >> File >> "/scratch/src/gcc-14.mine/contrib/testsuite-management/validate_failures.py", >> line 721, in Main >> retval = CompareBuilds() >> File >> "/scratch/src/gcc-14.mine/contrib/testsuite-management/validate_failures.py", >> line 622, in CompareBuilds >> actual = GetResults(sum_files) >> File >> "/scratch/src/gcc-14.mine/contrib/testsuite-management/validate_failures.py", >> line 466, in GetResults >> build_results.update(ParseSummary(sum_fname)) >> File >> "/scratch/src/gcc-14.mine/contrib/testsuite-management/validate_failures.py", >> line 405, in ParseSummary >> result = result_set.MakeTestResult(line, ordinal) >> File >> "/scratch/src/gcc-14.mine/contrib/testsuite-management/validate_failures.py", >> line 239, in MakeTestResult >> return TestResult(summary_line, ordinal, >> File >> "/scratch/src/gcc-14.mine/contrib/testsuite-management/validate_failures.py", >> line 151, in __init__ >> raise >> RuntimeError: No active exception to reraise >> >> >> The problem seems to be that gotools.sum does not mention any ".exp" >> files. >> >> $ grep "Running " gotools/gotools.sum >> Running cmd/go >> Running runtime >> Running cgo >> Running carchive >> Running cmd/vet >> Running embed >> $ grep -c "\.exp" gotools/gotools.sum >> 0 >> >> The .sum files looks like this: >> ---8<--- >> Test Run By foo on Tue Sep 26 14:46:48 CEST 2023 >> Native configuration is x86_64-foo-linux-gnu >> >> === gotools tests === >> >> Running cmd/go >> UNTESTED: TestAccidentalGitCheckout >> PASS: TestAlwaysLinkSysoFiles >> ... >> UNTESTED: TestParallelTest >> FAIL: TestScript >> ... >> ---8<--- >> >> May i ask you to have a look, please? >> >> TIA, > >
