nick-boss-tech opened a new pull request, #5062: URL: https://github.com/apache/solr/pull/5062
🤖 *AI text below* 🤖 *(posted on behalf of Nick Shanin)* https://issues.apache.org/jira/browse/SOLR-18523 ## What happens today `check` for `:solr:documentation` depends on `checkBrokenLinks`, a task that builds the full documentation site and then crawls it with `dev-tools/scripts/checkJavadocLinks.py` (277 lines) under python3, looking for broken links. The javadoc part of that validation already happens earlier and more cheaply: `gradle/java/javac.gradle` applies `-Xdoclint:all/protected` with `-Werror` to every compile, so an invalid `{@link}` in public or protected javadoc fails `compileJava` before any HTML exists. The crawl is a second, slower check of the same javadoc references, and it puts python3 on the `check` path. Lucene removed the equivalent crawl in apache/lucene#14905 for the same reason. ## What this change does - Deletes `dev-tools/scripts/checkJavadocLinks.py` and `gradle/validation/check-broken-links.gradle` (the `CheckBrokenLinksTask` class, the `:solr:documentation:checkBrokenLinks` task, and the root `checkBrokenLinks` alias), and its `apply from` line in the root build. - `:solr:documentation:check` now depends on the `documentation` task only on CI (`if (isCIBuild)` in `gradle/documentation/documentation.gradle`), the pattern Lucene used in #14905 including its CI gate: CI still builds the full site during `check`, so references only visible to the javadoc tool (for example in overview files) and problems in the documentation build itself still surface, while a local `check` no longer builds the documentation at all. - Behavior change, stated openly: links in the non-javadoc pages of the site (ref-guide and markdown pages, Changes.html) are no longer checked by anything. That trade, and the local `check` coverage trade from the CI gate, are the Choice below. ## Proof This is a build-wiring change; there is no unit test that fails on the base code. The proof is the DocLint measurement and the gates below, run 2026-10-08: the crawl removal at head `43f57c1b853` and the CI-gate verification at head `8a746512bba`, both on `main` at `d94c5eeb6b5`. - DocLint coverage on the base: full-tree `compileJava` with `reference` checks widened beyond the shipped `/protected` scope to all scopes passed with 0 failing references in every module. A planted bad `{@link}` fails `:solr:api:compileJava` under the shipped configuration (public scope) and under the widened one (public and private), so the compile-time check the crawl duplicates is real. - Lucene follow-ups audit: 13 Lucene changes after #14905 touching this machinery were dispositioned against this tree (audit record on the `assignment-18523-doclint` branch). One applied: the `check` to `documentation` edge in #14905 is gated on `isCIBuild`, and that gate is folded in here as commit `8a746512bba`. The other 12 do not apply to Solr's tree or are out of this change's scope, each with its reason recorded. - Task graph, at `8a746512bba`: root `check --dry-run` contains no `checkBrokenLinks` task, with or without `CI=true`. Scoped to `:solr:documentation:check --dry-run`, the graph contains `:solr:documentation:documentation` with `CI=true` and does not contain it without CI; that scoped pair pins the edge this change makes. (The whole-tree graph lists `documentation` in both variants through `:solr:solr-ref-guide`, which consumes the documentation site artifact; that edge is pre-existing and untouched.) - Execution, at `8a746512bba`, with Gradle daemons stopped between variants so `isCIBuild` re-reads the environment: with `CI=true`, `:solr:documentation:check -x test` is BUILD SUCCESSFUL in 5m 5s and runs `documentation` and the site assembly tasks, producing 8,225 HTML files (the documentation build directory was deleted first, so the tasks genuinely executed rather than reporting up-to-date). Without CI, the same command is BUILD SUCCESSFUL in 1m 19s, runs no `documentation` task, and produces no site. - Earlier gate, at `43f57c1b853`: `:solr:documentation:check -x test` BUILD SUCCESSFUL (40s; 8,225 HTML files in the built site). Root `check -x test` stops at `:rat` (license headers reported as unknown across the tree) and, with `-Pvalidation.rat.failOnError=false`, runs 10m 11s across the whole project and stops only at `:solr:solr-ref-guide:buildLocalAntoraSite`, where Antora rejects a linked git worktree as its content source. Both failures reproduce identically on unmodified main in the same worktree setup; neither is caused by this change. - The step this removes was measured at 4m 18s for the crawl alone, on top of the documentation build it depended on. ## A choice to check Two coverage trades come with this change; the implemented position on both is to follow Lucene. First, the crawl covers the entire documentation site, not only the javadoc tree: the docroot also contains the ref-guide and markdown pages and Changes.html. DocLint validates javadoc `{@link}` references at compile time and checks nothing in those other pages. The failure in SOLR-18119's report is of the non-javadoc kind (a ref-guide page linking to a missing Changes.html). This trade is not Solr diverging from Lucene: the crawl Lucene deleted in #14905 also read `file(project.docroot)`, the whole site, so Lucene accepted the same site-wide trade. Implemented position: remove the crawl anyway, because the javadoc coverage is now compile-time and the crawl is the remaining python3 dependency in `check`. The alternatives are to keep a reduced crawl for the non-javadoc pages only, which keeps a checker (Python, or the Java port in #5061) alive for that slice, or to land this removal and track site-wide link coverage as its own issue. Second, the CI gate means a local `check` no longer builds the documentation at all. Before this change, every local `check` ran `documentation` (the crawl depended on it), so overview.html reference validation and the missing-doclet level checks, which ride on `renderSiteJavadoc`, ran locally too; now they run only when `isCIBuild` is set (`CI`, or `JENKINS_*` / `HUDSON_*` variables). Lucene made the same call in #14905 and kept the gate in its later Java port (#15350). Implemented position: gate the edge, accepting faster local `check` runs in exchange for CI-only coverage of those checks. The alternative is the unconditional edge, which keeps the local coverage at the cost of a full documentation build on every local `check`. Were both trades the right call? ## Limits - The `missing` DocLint group stays disabled (`-Xdoclint:-missing` in `gradle/java/javac.gradle` is untouched). Measured on the base at the same `/protected` scope: enabling it reports 22,710 diagnostics across `compileJava` (15,450 public or protected elements with no javadoc comment, 3,079 missing `@param`, 1,849 missing `@return`, 1,056 undocumented default constructors, 711 missing `@throws`, 565 tag-only or empty comments). That is a tree-wide javadoc completion effort, not part of this change; happy to open a follow-up ticket for it. - The all-scopes reference measurement covers `compileJava`. `compileTestJava` carries the same DocLint configuration but was not measured separately; the crawl never checked test javadoc either. Changelog: `changelog/unreleased/SOLR-18523.yml` ### AI assistance AI agents assisted with research, implementation, review, and drafting. Nick Shanin directed the work and takes responsibility for this contribution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
