On Wed, Oct 11, 2023 at 08:17:49AM -0600, Jeff Law wrote:
>
>
> On 10/11/23 08:10, Richard Earnshaw (lists) wrote:
> > On 11/10/2023 14:56, Jeff Law wrote:
> > >
> > >
> > > On 10/11/23 04:39, Florian Weimer wrote:
> > > > I've started to look at what it is required to convert the testsuite to
> > > > C99 (without implicit ints, without implicit function declarations, and
> > > > a few other legacy language features).
> > > I bet those older tests originating from c-torture will be a bit painful.
> > > Torbjorn liked having them minimized, to the point of squashing out
> > > nearly everything he considered extraneous. I'd bet many of those older
> > > tests are going to need lots of changes.
> > >
> >
> > I've often wondered just how much of the original c-torture suite is still
> > relevant today. Most of those tests were written at a time when the
> > compiler expanded tree directly into RTL and I suspect that today the tests
> > never get even close to tickling the original bug they were intended to
> > validate.
> I'm sure it's a mixed bag. Some I do see pop up regularly during
> development testing.
>
> The real way to try and answer that question would be with gcov. Something
> like take an existing coverage report, run test and see if it added any
> additional coverage. If not, flag it as potentially irrelevant. Do this on
> x86 or aarch64.
>
> Given that list of potentially irrelevant tests, then look at them from a
> target standpoint, potentially testing with the same methodology on several
> targets, perhaps a standardized set meant to cover 16->64 bit targets,
> big/little endian and a few other key features.
I think over the years we've seen tons of cases where a test written for one
problem at some point is able to trigger some completely different problem a
few years ago. So, I don't think most of the tests are irrelevant.
Jakub
