On Wed, Aug 21, 2024 at 2:01 AM David Malcolm <dmalc...@redhat.com> wrote: > > On Tue, 2024-08-20 at 11:49 +0200, Richard Biener wrote: > > On Thu, Aug 15, 2024 at 8:13 PM David Malcolm <dmalc...@redhat.com> > > wrote: > > > > > > Here's v3 of my patch kit for "libdiagnostics", which makes GCC's > > > diagnostics subsystem available as a shared library; see: > > > https://gcc.gnu.org/wiki/libdiagnostics > > > > So this is to make use of this from gas? > > One of the clients of the library would be gas, yes (although > optionally, to avoid complicating bootstrap of binutils). > > However other clients would be possible, including those that are not > under the "GNU toolchain" umbrella (provided that they can be linked > against GPLv3 code). > > For example, I've been experimenting with Python bindings, which would > allow Python scripts to reuse GCC's diagnostics code (e.g. for SARIF, > fix-it hints, etc). I've also been experimenting with IDE integration > (see PR 115970), and it would be nice if users of libdiagnostics got > that "for free". > > Another example of client code is the sarif-replay tool in patch 7, as > a sarif consumer (if nothing else, writing this has exposed various > bugs in our existing SARIF-writing code). The analyzer's integration > test suite generates 10s of thousands of .sarif files, so having > tooling to work with them is "scratching my own itch". > > > Is the plan to move > > sources (and dependences) to the toplevel then, possibly > > building a static convenience lib for GCCs use? > > The problem with moving things from "gcc" to, say, a new subdirectory > of the top level source directory is that our "gcc" subdirectory has a > lot of support code that would also need refactoring/moving. Off the > top of my head: > - selftest framework > - DejaGnu .exp stuff below gcc/testsuite/lib > - C++11 support, such as our "make-unique.h" > - libcpp: the big one: diagnostics uses libcpp > - probably some configure/Makefile.in entanglements > > Fixing the above would be a major task.
Yeah - I was wondering about dependences ... > So the patch kit punts on this by adding/moving stuff within "gcc" (and > requiring an opt-in via --enable-libdiagnostics). OK, fair. > > > > Note I'm missing documentation (which is probably there > > in the libdiagnostics.h header); an addition to sourcebuild.texi > > might be nice at least and documenting --enable-libdiagnostics > > in install.texi. > > The libdiagnostics.h header has comments, but, yes, I should probably > add docs similar to that of libgccjit.h (tutorial and API reference). > I'll do that for the next iteration of the patch. > > Patch 2 of the kit documents --enable-libdiagnostics in install.texi. > I'll add notes to sourcebuild.texi in the next iteration of the patch. > > > Thanks; hope the above makes sense. Sure. Note I'm not super-happy with adding maintainance burden on the GCC side for external users when the component is so deeply interwinded with GCC internals. But I won't object ;) Richard. > Dave > > > > > > New in v3: > > > * it bootstraps and pass regression tests > > > * I added an opt-in configure flag: --enable-libdiagnostics, which > > > must be enabled to build it (along with --enable-host-shared) > > > * a new "sarif-replay" command-line tool that takes .sarif files > > > and replays the diagnostics within them as if they were GCC > > > diagnostics, in GCC's textual format (i.e. GCC as a SARIF > > > *consumer*, > > > as well as producer). This is implemented on top of > > > libdiagnostics > > > hence I've been "eating my own dogfood" > > > * support for execution paths in libdiagnostics API > > > * lots of fixes > > > > > > Patch 1 has libdiagnostic.h, the public header file > > > Patch 2 has the implementation > > > Patch 3 has the C++ wrapper API I added in v2 > > > Patch 4 has a refactoring of gcc-dg.exp I needed for patch 5. > > > Patch 5 has the testsuite for libdiagnostics itself > > > Patch 6 implements JSON parsing support > > > Patch 7 implements the sarif-replay command-line tool, and its > > > testsuite, exercising various valid, invalid, and malformed > > > input files. > > > > > > Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu, > > > both with and without --enable-libdiagnostics. > > > With --enable-libdiagnostics the patch kit has this effect: > > > > > > # of .sum files: 20->22 (+2) > > > FAIL: 110 > > > PASS: 617481->617679 (+198) 100.03% > > > XFAIL: 4512 > > > XPASS: 13 > > > UNTESTED: 136 > > > UNSUPPORTED: 8058 > > > > > > where the two new .sum files are: > > > > > > BUILD/gcc/testsuite/libdiagnostics/libdiagnostics.sum: > > > PASS: 132 tests > > > > > > BUILD/gcc/testsuite/sarif-replay/sarif-replay.sum: > > > PASS: 66 tests > > > > > > OK for trunk? > > > > > > David Malcolm (7): > > > libdiagnostics v3: header > > > libdiagnostics v3: implementation > > > libdiagnostics v3: add C++ wrapper API > > > testsuite: move dg-test cleanup code from gcc-dg.exp to its own > > > file > > > libdiagnostics v3: test suite > > > json: add json parsing support > > > libdiagnostics: add a "sarif-replay" command-line tool [PR96032] > > > > > > configure | 42 + > > > configure.ac | 35 + > > > contrib/regenerate-sarif-spec-index.py | 60 + > > > gcc/Makefile.in | 191 +- > > > gcc/configure | 26 +- > > > gcc/configure.ac | 16 + > > > gcc/diagnostic-event-id.h | 6 + > > > gcc/doc/install.texi | 13 + > > > gcc/json-parsing.cc | 2394 > > > +++++++++++++++++ > > > gcc/json-parsing.h | 113 + > > > gcc/json.cc | 2 +- > > > gcc/json.h | 122 +- > > > gcc/libdiagnostics++.h | 595 ++++ > > > gcc/libdiagnostics.cc | 1652 ++++++++++++ > > > gcc/libdiagnostics.h | 691 +++++ > > > gcc/libdiagnostics.map | 72 + > > > gcc/libsarifreplay.cc | 1747 ++++++++++++ > > > gcc/libsarifreplay.h | 59 + > > > gcc/sarif-replay.cc | 239 ++ > > > gcc/sarif-spec-urls.def | 496 ++++ > > > gcc/selftest-run-tests.cc | 1 + > > > gcc/selftest.h | 1 + > > > gcc/testsuite/lib/dg-test-cleanup.exp | 116 + > > > gcc/testsuite/lib/gcc-dg.exp | 106 +- > > > gcc/testsuite/lib/sarif-replay-dg.exp | 90 + > > > gcc/testsuite/lib/sarif-replay.exp | 204 ++ > > > .../libdiagnostics.dg/libdiagnostics.exp | 296 ++ > > > gcc/testsuite/libdiagnostics.dg/sarif.py | 23 + > > > gcc/testsuite/libdiagnostics.dg/test-dump.c | 69 + > > > .../libdiagnostics.dg/test-error-c.py | 54 + > > > .../test-error-with-note-c.py | 50 + > > > .../libdiagnostics.dg/test-error-with-note.c | 74 + > > > .../libdiagnostics.dg/test-error-with-note.cc | 55 + > > > gcc/testsuite/libdiagnostics.dg/test-error.c | 59 + > > > gcc/testsuite/libdiagnostics.dg/test-error.cc | 47 + > > > .../libdiagnostics.dg/test-fix-it-hint-c.py | 46 + > > > .../libdiagnostics.dg/test-fix-it-hint.c | 81 + > > > .../libdiagnostics.dg/test-fix-it-hint.cc | 74 + > > > .../libdiagnostics.dg/test-helpers++.h | 28 + > > > .../libdiagnostics.dg/test-helpers.h | 72 + > > > .../libdiagnostics.dg/test-labelled-ranges.c | 69 + > > > .../libdiagnostics.dg/test-labelled-ranges.cc | 64 + > > > .../libdiagnostics.dg/test-labelled-ranges.py | 48 + > > > .../test-logical-location-c.py | 37 + > > > .../libdiagnostics.dg/test-logical-location.c | 79 + > > > .../libdiagnostics.dg/test-metadata-c.py | 45 + > > > .../libdiagnostics.dg/test-metadata.c | 61 + > > > .../test-multiple-lines-c.py | 83 + > > > .../libdiagnostics.dg/test-multiple-lines.c | 76 + > > > .../libdiagnostics.dg/test-no-column-c.py | 35 + > > > .../libdiagnostics.dg/test-no-column.c | 52 + > > > .../test-no-diagnostics-c.py | 42 + > > > .../libdiagnostics.dg/test-no-diagnostics.c | 25 + > > > .../test-note-with-fix-it-hint-c.py | 54 + > > > .../test-note-with-fix-it-hint.c | 69 + > > > .../test-text-sink-options.c | 59 + > > > .../libdiagnostics.dg/test-warning-c.py | 54 + > > > .../test-warning-with-path-c.py | 108 + > > > .../test-warning-with-path.c | 125 + > > > .../libdiagnostics.dg/test-warning.c | 65 + > > > .../test-write-sarif-to-file-c.py | 55 + > > > .../test-write-sarif-to-file.c | 55 + > > > .../test-write-text-to-file.c | 47 + > > > .../2.1.0-invalid/3.1-not-an-object.sarif | 6 + > > > .../3.11.11-malformed-placeholder.sarif | 15 + > > > ...1-missing-arguments-for-placeholders.sarif | 14 + > > > ...ot-enough-arguments-for-placeholders.sarif | 14 + > > > .../2.1.0-invalid/3.13.2-no-version.sarif | 6 + > > > .../3.13.2-version-not-a-string.sarif | 6 + > > > .../2.1.0-invalid/3.13.4-bad-runs.sarif | 7 + > > > .../2.1.0-invalid/3.13.4-no-runs.sarif | 6 + > > > .../3.13.4-non-object-in-runs.sarif | 7 + > > > .../2.1.0-invalid/3.27.10-bad-level.sarif | 25 + > > > .../2.1.0-unhandled/3.27.10-none-level.sarif | 25 + > > > .../2.1.0-valid/error-with-note.sarif | 34 + > > > .../2.1.0-valid/escaped-braces.sarif | 17 + > > > .../2.1.0-valid/null-runs.sarif | 2 + > > > .../2.1.0-valid/signal-1.c.sarif | 193 ++ > > > .../2.1.0-valid/spec-example-1.sarif | 15 + > > > .../2.1.0-valid/spec-example-2.sarif | 73 + > > > .../2.1.0-valid/spec-example-3.sarif | 65 + > > > .../2.1.0-valid/spec-example-4.sarif | 766 ++++++ > > > .../2.1.0-valid/tutorial-example.sarif | 117 + > > > gcc/testsuite/sarif-replay.dg/dg.exp | 46 + > > > .../malformed-json/array-missing-comma.sarif | 6 + > > > .../array-with-trailing-comma.sarif | 6 + > > > .../malformed-json/bad-token.sarif | 6 + > > > .../malformed-json/object-missing-comma.sarif | 7 + > > > .../object-with-trailing-comma.sarif | 6 + > > > libatomic/testsuite/lib/libatomic.exp | 1 + > > > libgomp/testsuite/lib/libgomp.exp | 1 + > > > libitm/testsuite/lib/libitm.exp | 1 + > > > libphobos/testsuite/lib/libphobos-dg.exp | 1 + > > > libstdc++-v3/testsuite/lib/libstdc++.exp | 1 + > > > libvtv/testsuite/lib/libvtv.exp | 1 + > > > 95 files changed, 12912 insertions(+), 108 deletions(-) > > > create mode 100644 contrib/regenerate-sarif-spec-index.py > > > create mode 100644 gcc/json-parsing.cc > > > create mode 100644 gcc/json-parsing.h > > > create mode 100644 gcc/libdiagnostics++.h > > > create mode 100644 gcc/libdiagnostics.cc > > > create mode 100644 gcc/libdiagnostics.h > > > create mode 100644 gcc/libdiagnostics.map > > > create mode 100644 gcc/libsarifreplay.cc > > > create mode 100644 gcc/libsarifreplay.h > > > create mode 100644 gcc/sarif-replay.cc > > > create mode 100644 gcc/sarif-spec-urls.def > > > create mode 100644 gcc/testsuite/lib/dg-test-cleanup.exp > > > create mode 100644 gcc/testsuite/lib/sarif-replay-dg.exp > > > create mode 100644 gcc/testsuite/lib/sarif-replay.exp > > > create mode 100644 > > > gcc/testsuite/libdiagnostics.dg/libdiagnostics.exp > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/sarif.py > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-dump.c > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error-c.py > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error- > > > with-note-c.py > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error- > > > with-note.c > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error- > > > with-note.cc > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error.c > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error.cc > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-fix-it- > > > hint-c.py > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-fix-it- > > > hint.c > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-fix-it- > > > hint.cc > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test- > > > helpers++.h > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-helpers.h > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-labelled- > > > ranges.c > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-labelled- > > > ranges.cc > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-labelled- > > > ranges.py > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-logical- > > > location-c.py > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-logical- > > > location.c > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-metadata- > > > c.py > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-metadata.c > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-multiple- > > > lines-c.py > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-multiple- > > > lines.c > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-no-column- > > > c.py > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-no- > > > column.c > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-no- > > > diagnostics-c.py > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-no- > > > diagnostics.c > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-note-with- > > > fix-it-hint-c.py > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-note-with- > > > fix-it-hint.c > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-text-sink- > > > options.c > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-warning- > > > c.py > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-warning- > > > with-path-c.py > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-warning- > > > with-path.c > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-warning.c > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-write- > > > sarif-to-file-c.py > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-write- > > > sarif-to-file.c > > > create mode 100644 gcc/testsuite/libdiagnostics.dg/test-write- > > > text-to-file.c > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0- > > > invalid/3.1-not-an-object.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0- > > > invalid/3.11.11-malformed-placeholder.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0- > > > invalid/3.11.11-missing-arguments-for-placeholders.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0- > > > invalid/3.11.11-not-enough-arguments-for-placeholders.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0- > > > invalid/3.13.2-no-version.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0- > > > invalid/3.13.2-version-not-a-string.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0- > > > invalid/3.13.4-bad-runs.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0- > > > invalid/3.13.4-no-runs.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0- > > > invalid/3.13.4-non-object-in-runs.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0- > > > invalid/3.27.10-bad-level.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0- > > > unhandled/3.27.10-none-level.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0- > > > valid/error-with-note.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0- > > > valid/escaped-braces.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-valid/null- > > > runs.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0- > > > valid/signal-1.c.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-valid/spec- > > > example-1.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-valid/spec- > > > example-2.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-valid/spec- > > > example-3.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-valid/spec- > > > example-4.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0- > > > valid/tutorial-example.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/dg.exp > > > create mode 100644 gcc/testsuite/sarif-replay.dg/malformed- > > > json/array-missing-comma.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/malformed- > > > json/array-with-trailing-comma.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/malformed- > > > json/bad-token.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/malformed- > > > json/object-missing-comma.sarif > > > create mode 100644 gcc/testsuite/sarif-replay.dg/malformed- > > > json/object-with-trailing-comma.sarif > > > > > > -- > > > 2.26.3 > > > > > >
