On Thu, 13 Mar 2025, Sam James wrote:
> Simon Sobisch <simonsobi...@gnu.org> writes:
>
> > Thanks for your work on adding a testsuite. Can you please explain why
> > you do this when a complete testsuite exists in autoconf (autotest)
> > format (which roots back to decade of work in GnuCOBOL, with all
> > copyrights for that already with the FSF)?
> >
>
> I don't think any of us were aware of it ("we" being "the general GCC
> developer community", not the COBOL folks, for the purposes of this
> email) until yesterday when richi mused about it on IRC maybe existing
> and we went looking out of curiosity.
>
> I agree that having that testsuite integrated would be fantastic.
>
> > Is the existence of this in upstream [1] just unknown (because it was
> > not part of the initial patches [for reasons I not understood])?
> >
>
> I would've personally liked to see the NIST testsuite integration at
> least in the initial patches, but it is what it is. I don't think the
> GnuCOBOL testsuite was brought up at all (and I think most of us weren't
> aware of it) in the patch upstreaming discussions.
>
> Now that we *are* aware of it, it seems desirable to have for sure.
>
> > Is the format such a big issue (note: previous discussions elaborated
> > "a test suite is very important and other frontends also use a
> > framework other than dejagnu)?
> >
> > If dejagnu is the way to go:
> >
> > * Shouldn't there be deprecation of autotest in autoconf (of course
> > only if that preference is also outside of gcc)?
>
> It's a GCC / GNU toolchain-only preference because it allows easily
> doing cross + simulator testing, and all of our tools are used to its
> format.
That's indeed the main reason.
> It's definitely not perfect. Years ago (way before I followed GCC),
> there was talk of replacing dejagnu, just efforts failed.
>
> >
> > * Shouldn't there be a (at least semi automated) script / migration
> > tool (at least for this specific time in place to convert the "UAT"
> > once into dejagnu format)?
>
> Yes. Having testsuite integration is seen as critical at this
> point. richi just wanted to present this as a non-COBOL person to give
> us something to play with.
Yes, and to give people familiar with how GCC tests are done a place
to put regression tests going forward.
I do think that integrating the testsuites the COBOLworx folks have
is important and of course integrating tests from GNU Cobol is desirable
as well. Whether we can or want to integrate tests based on autotest
is another question - I'd probably avoid that, even as short-term
solution, as such tend to stay forever.
What would be nice is to have a common separate test harness you can
test an installed compiler against - I'm not sure whether the GNU
Cobol test harness or the COBOLworx one qualifies here. The NIST
one probably does, but it seems to require "plumbing" that's not
part of NIST and that, in implementation, might differ from GNU Cobol
to COBOLworx.
Richard.
> >
> >
> >
> > Thanks for giving me some context on this,
> > Simon
> >
> >
> > [1]:
> > https://gitlab.cobolworx.com/COBOLworx/gcc-cobol/-/tree/master+cobol/gcc/cobol/UAT
>
> thanks,
> sam
>
--
Richard Biener <rguent...@suse.de>
SUSE Software Solutions Germany GmbH,
Frankenstrasse 146, 90461 Nuernberg, Germany;
GF: Ivo Totev, Andrew McDonald, Werner Knoblich; (HRB 36809, AG Nuernberg)
