Am 07.04.2025 um 09:30 schrieb Richard Biener:
On Mon, Apr 7, 2025 at 9:00 AM Simon Sobisch <simonsobi...@gnu.org> wrote:
My question stands on integrating COBOLworx' UAT as-is for now
(Copyright is all on FSF; built automatically [it is autoconf, which is
a requirement for VCS checkouts], possibly also hooked into the current
test target) - with the goal to get rid of UAT later (next GCC version,
not GCC 15).
There's also the question about integrating NIST into GCC upstream -
that is a subfolder and would only be executed upon explicit call by
maintainers (newcob.val / newcob.val.gz may be either included in VCS or
even downloaded manually...).
As I repeatedly said I'd welcome a test harness like Ada ACATS for running
the NIST testsuite plus a contrib/download_cobol_nist script that downloads
the NIST file and prepares it for use. I'd suggest to, similar as with ACATS,
have a separate make target for testing (but still invoked with make check,
when present).
Sounds good. As it is a single curl operation that can also be done with
make in that subfolder, do we still need a separate download script?
I understand that in any case the test harness would check for the
newcob.val file existing (builddir only is fine, right?), and if it is,
then execute a `${MAKE} -C builddir/...test.../nist).
If Jim doesn't find the time to do this (please respond on this), I can
prepare a patch (contributing mostly COBOLworx' work for that setup and
config).
As for UAT, I understand it's work in progress to get that converted to dejagnu?
It is, but full UAT will take weeks, if not months, as far as I've
understood Bob. I feel the urge to have his time spend on other things
than a conversion (which _does_ provide additional benefit like testing
with more configurations and be better included in the rest of GCC's
tests), as GCC15 is near and the amount of things to don don't get less).
But whatever you guys decide will happen, I mostly wanted to raise my
concern.
With UAT, gcobol would have MUCH more test coverage directly for
everyone, with NIST developers would have the chance to run "what is not
disabled" from that testsuite for bigger changes like the
FLOAT_128/libmath adjustment and when working on a new target.
Both parts are already in the COBOLworx repo and work, can be used
directly to check for regressions and the move from UAT to dejagnu can
still be done after the increasing pile of bugs (which, as a COBOL
programmer I partly find quite severe) and possibly some feature
requests (especially around huge codegen) are taken care by the "rare
resources" Bob and Jim.
Concerning NIST: please take care to not get on the same low level like
COBOL-IT and others, claiming gcobol passes NIST - it doesn't (no
current compiler does pass all modules, and I think GnuCOBOL is the
single one that nearly passes everything [and is able to at least parse
the parts that are disabled - around the COMMUNICATION module which was
obsolete in COBOL85 and was kind of resurrected by COBOL2023's Message
Control System [MCS]).
Simon
