Martin Jambor <mjam...@suse.cz> writes: > Hi, > > On Sat, Jul 15 2023, FX Coudert via Gcc wrote: >> Hi, >> >> I am finding it very hard to reliably compare test results and regressions >> with the very large number of gcc.dg/guality test failures that are >> apparently the new norm on x86_64-linux: more than one hundred on 13.1, and >> several hundreds on 14. Is there any on-going discussion about this? >> >> I mean, from an almost-external point of view, these tests should probably >> be xfail'ed and a PR opened against them to reenable them. >> > > As far as I understand it, the main problem is that it is not really > possible to XFAIL a test for one combination of options (say "-O2 > -flto") and not others.
FWIW, we do do that for aarch64*. The principle is that if anyone wants to work on improving debug quality, the XFAILs in the testsuite are just as good a reference as bugzilla (unless the bugzilla PR has specific analysis). But it is still dependent on the default configuration and on the version of GDB used. It gives zero failures on my set-up, and seems to do the same for some other gcc-testresults@ emails I've seen. But it probably won't for everyone. I tend to update the XFAIL lists in stage 3/4, when no major changes are expected. Having the XFAILs doesn't really change the workflow. It's still necessary to do a before-after comparison. But having fewer XPASSes & FAILs makes it less likely that something gets missed. It might even encourage people to take guality regressions more seriously :) Before, there were so many XPASSes and FAILs in the "before" log that it was easy think "well, what's one more?" or "this thing looks generally broken, so I'll just ignore it". Richard
