On Mon, Apr 7, 2025 at 9:41 AM Simon Sobisch <simonsobi...@gnu.org> wrote:
>
>
>
> Am 07.04.2025 um 09:30 schrieb Richard Biener:
> > On Mon, Apr 7, 2025 at 9:00 AM Simon Sobisch <simonsobi...@gnu.org> wrote:
> >>
> >> My question stands on integrating COBOLworx' UAT as-is for now
> >> (Copyright is all on FSF; built automatically [it is autoconf, which is
> >> a requirement for VCS checkouts], possibly also hooked into the current
> >> test target) - with the goal to get rid of UAT later (next GCC version,
> >> not GCC 15).
> >>
> >> There's also the question about integrating NIST into GCC upstream -
> >> that is a subfolder and would only be executed upon explicit call by
> >> maintainers (newcob.val / newcob.val.gz may be either included in VCS or
> >> even downloaded manually...).
> >
> > As I repeatedly said I'd welcome a test harness like Ada ACATS for running
> > the NIST testsuite plus a contrib/download_cobol_nist script that downloads
> > the NIST file and prepares it for use. I'd suggest to, similar as with
> > ACATS,
> > have a separate make target for testing (but still invoked with make check,
> > when present).
>
> Sounds good. As it is a single curl operation that can also be done with
> make in that subfolder, do we still need a separate download script?
I think that would be better, yes.
> I understand that in any case the test harness would check for the
> newcob.val file existing (builddir only is fine, right?), and if it is,
> then execute a `${MAKE} -C builddir/...test.../nist).
Yes. I agree that the actual testing should ideally be done via dejagnu,
but OTOH all the extracting part can be done on the host and thus in
a script/Makefile.
> If Jim doesn't find the time to do this (please respond on this), I can
> prepare a patch (contributing mostly COBOLworx' work for that setup and
> config).
>
> > As for UAT, I understand it's work in progress to get that converted to
> > dejagnu?
>
> It is, but full UAT will take weeks, if not months, as far as I've
> understood Bob. I feel the urge to have his time spend on other things
> than a conversion (which _does_ provide additional benefit like testing
> with more configurations and be better included in the rest of GCC's
> tests), as GCC15 is near and the amount of things to don don't get less).
>
> But whatever you guys decide will happen, I mostly wanted to raise my
> concern.
Note the set of Cobol tests in GCC 15 does not need to stay fixed, tests
can be added there even after the GCC 15.1 release. Mind GCC 15 will
live for quite some time.
Richard.
> >> With UAT, gcobol would have MUCH more test coverage directly for
> >> everyone, with NIST developers would have the chance to run "what is not
> >> disabled" from that testsuite for bigger changes like the
> >> FLOAT_128/libmath adjustment and when working on a new target.
> >>
> >> Both parts are already in the COBOLworx repo and work, can be used
> >> directly to check for regressions and the move from UAT to dejagnu can
> >> still be done after the increasing pile of bugs (which, as a COBOL
> >> programmer I partly find quite severe) and possibly some feature
> >> requests (especially around huge codegen) are taken care by the "rare
> >> resources" Bob and Jim.
> >>
> >> Concerning NIST: please take care to not get on the same low level like
> >> COBOL-IT and others, claiming gcobol passes NIST - it doesn't (no
> >> current compiler does pass all modules, and I think GnuCOBOL is the
> >> single one that nearly passes everything [and is able to at least parse
> >> the parts that are disabled - around the COMMUNICATION module which was
> >> obsolete in COBOL85 and was kind of resurrected by COBOL2023's Message
> >> Control System [MCS]).
>
> Simon
>
>
