Jakub Jelinek <ja...@redhat.com> writes:
> On Thu, Jan 29, 2015 at 11:11:12PM -0700, Jeff Law wrote:
>> > Linux/x86_64 Solaris 11/x86 Solaris
>> > 11/SPARC
>> > (Fedora 20)
>> >
>> >gcc.dg/guality:
>> >
>> ># of expected passes 6490 6500 5489
>> ># of unexpected failures 191 171 802
>> ># of unexpected successes 61 66 73
>> ># of expected failures 35 30 23
>> ># of unsupported tests 257 267 383
>> >
>> >g++.dg/guality:
>> >
>> ># of expected passes 128 128 118
>> ># of unexpected failures 6 10 10
>> ># of unsupported tests 34 30 40
>> Could we select a version of gdb as a reference version, then xfail those
>> tests which don't work with that reference version on each platform to cut
>> down the noise? Obviously we bump the reference version, probably as we
>> close stage1 or stage3 development?
>
> The biggest problem is that what fails and what does not varries between
> targets and between optimization levels. Right now we have no way to xfail
> test XYZ for -Os on x86_64-linux and for -O2 and -O3 on i686-linux ia32, and
> the lists would become very large. Some tests in guality are xfaileded just
> in case, even when they actually XPASS on many targets.
Right: while we could add such a facility (the current xfail selector
support is already beyond what vanilla DejaGnu provides), it would
quickly become very unwieldly. The best we can do right now is to use
dg-xfail-run-if more precisely for tests that fail to execute. For
gdb-test failures, the only option would be to dg-skip-if the affected
tests per target and optimization level; quite large a hammer in some
cases. I've started looking at this, but it's incredibly tedious.
> The way to look for regressions in the guality area, at least as I do it
> regularly, is just compare test_summary results.
> If we'd disable this by default, I'm sure our debug quality would sink very
> quickly.
I'm not sure: my impression is that nobody really cared about the
guality tests in quite some time. Comparing test results might work
resonably, looking over the output if the vast majority of it is just
guality failures becomes infeasible. I may end up running with
GUALITY_GDB_NAME=/bin/true ;-(
Rainer
--
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University
