janhoy opened a new pull request, #5323: URL: https://github.com/apache/solr/pull/5323
https://issues.apache.org/jira/browse/SOLR-18642 > [!NOTE] > This is a proof of concept. Parts of it are expected to be split into their own PRs before anything lands — the `bin/solr.cmd` change below first among them — and each bug the suite uncovered gets its own JIRA rather than a fix in here. The JIRA covers what CATS is. This is how it is built and how to review it. <img width="980" height="500" alt="cats-overview" src="https://github.com/user-attachments/assets/8bc4133d-94a0-411c-bf36-13cdae88f641"; /> ## Layout | | | | --- | --- | | `solr/test-framework/src/java/org/apache/solr/shell/` | the harness: 20 files, ~4.3k lines | | `solr/packaging/src/test/org/apache/solr/shell/` | 30 suites, ~4.4k lines, replacing the 29 `.bats` files | | `solr/packaging/build.gradle`, `.github/workflows/cats-test.yml` | the `catsTests*` tasks, and a bash + cmd.exe CI matrix | `SolrShellTestCase` is the base class, `session/CliSession` runs commands and owns the rails, `env/` resolves directories and ports. ## How it works - **Per-class isolation is what buys the parallelism**: each class gets its own `SOLR_HOME`, `SOLR_PID_DIR`, logs directory and four ports (stop at `P-1000`, ZooKeeper at `P+1000`), claimed from a counter file under a `FileChannel.lock()` and kept below every platform's ephemeral port floor. **NB:** bin/solr.cmd needs a fix to respect `SOLR_PID_DIR` - **Nodes are found by process scan, not pid files** — a `solr.process.scan.marker` sysprop that `SolrProcessManager` filters on, which behaves identically on both platforms and survives a `SOLR_HOME` wipe. - **The harness resolves `${NAME}` before any shell sees it** (bash expands `$NAME`, cmd.exe wants `%NAME%`), failing on an unknown name instead of substituting empty. Non-portable constructs — shell operators, `$(`, backticks, `/dev/null`, ~45 Unix-only tools — fail at the call site naming the fluent method that replaces them; `@EnabledOnShell` lifts the rails per test. - **`run()` asserts success**, `runFail()` a non-zero status, `tryRun()` nothing; making success the default found 13 sites that had been ignoring a non-zero exit. `curl`, `jq`, `openssl` and `docker` became Java and Testcontainers, so no test needs an external tool or the network. - **A failure reads as a shell session**: every command with its exit code and output, one `FAILED:` line, a `SOLR_HOME` snapshot, and pre/post-flight port checks that pin a leak on the class that caused it. ## Running and reviewing ```bash ./gradlew -p solr/packaging catsTests # suites; no Docker, no network ./gradlew -p solr/packaging catsTests --tests "*ZkTest" # one class ./gradlew -p solr/packaging catsTestsAll # + rolling upgrade, builds an image ``` `catsTestsPicocli`/`catsTestsCommonsCli` pin the CLI implementation, `-Pcats.tests.jvms=N` sets the fork count, and the same commands run under cmd.exe. Output and per-test `transcript.log` land under `solr/packaging/build/cats-test-output*/`. Reading order: `CliSession` and `SolrShellTestCase`, then `HarnessEnv`/`PortBlocks` for the isolation model, then `ZkTest.java` beside `test_zk.bats`. The BATS suites still pass, so the two can be compared directly. ## TODO - [ ] Move the `bin/solr.cmd` `SOLR_PID_DIR` change to its own JIRA and PR, with a changelog entry and tests, then drop it from this branch - [ ] File a JIRA per bug the suite found — 18 so far, none fixed here - [ ] Triage the remaining Windows failures, confirming each stays red deliberately - [ ] Create the `cats-tests` GitHub label that `.github/labeler.yml` already references - [ ] Consider splitting `SslTest` and `StartSolrTest`, the parallel critical path, for roughly a third off the wall clock - [ ] Delete the BATS suites once the ports have proven themselves on Windows 🤖 Developed almost entirely by Claude Code. I have not yet reviewed all of it myself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
