On 8/30/25 12:22, David Malcolm wrote:
On Sat, 2025-08-30 at 10:49 -0600, Rob Savoye via Gcc wrote:
On 8/29/25 10:47 PM, Jacob Bachmeyer wrote:
There is an option for XML logs that is currently a bit "half-
baked"
and, as far as I can tell, what it
Sorry about that, I'm probably the only one that ever used it. The
entire idea of the .sum files was so the output could be easily
diff'd,
XML potentially being somewhat better cause at least it was
structured.
The idea of using a JSON file is probably better than fixing the
minimal
XML support. Or support both.
I mentioned this in a different branch of this discussion, but perhaps
the JSON format to use for test results could be SARIF:
https://sarif.info/
in that it has already has an extensive "vocabulary" for describing
results, related source locations, comparison against a baseline, etc.
For example, the 3.27.9 "kind" property maps quite nicely to PASS,
FAIL, UNSUPPORTED (and perhaps XPASS and XFAIL using the
"baselineState" property).
SARIF seems at first glance to be a very bad match to DejaGnu's data
model, or to testsuite results generally.
The problem, as I gather from a quick reading, is that the 3.27.9 "kind"
property is attached to a 3.27 "result" object, which is specific to the
evaluation of a rule. Further, it appears that those rules must be
declared elsewhere in the SARIF log and it seems that "rules" would
correspond to tests in DejaGnu.
Since the DejaGnu framework does not know the full set of tests in a
testsuite, nor impose the stability demanded in the SARIF spec, there
seems to be no reasonable way to map DejaGnu results to SARIF that would
not severely "twist" SARIF and badly confuse any unsuspecting SARIF
processors presented with the result.
Disclosure: I'm on the SARIF Technical Committee, and maintain GCC's
support for SARIF output for its diagnostics and some other SARIF
tooling, so I clearly have an interest here!
Dave
-- Jacob
