Jan Høydahl created SOLR-18642:
----------------------------------
Summary: Replace BATS with java-native shell test harness
Key: SOLR-18642
URL: https://issues.apache.org/jira/browse/SOLR-18642
Project: Solr
Issue Type: Improvement
Components: cli, test-framework
Reporter: Jan Høydahl
Assignee: Jan Høydahl
Today we have BATS for testing {{bin/solr}} and realistic CLI type tests in
bash. We have nothing similar for Windows. SOLR-17508 explored Pester, but it
went nowhere.
This issue proposes *CATS* — a Cross-platform Automated Testing System — a
Java-native harness for shell testing, so such tests run under JUnit like any
other test, in parallel, on every platform. The name follows BATS (Bash
Automated Testing System) deliberately: same idea, one letter and one platform
restriction fewer.
h3. The DSL
Pure Java with convenience methods, so a ported BATS line reads almost
unchanged:
{code:java}
@Test
public void aNodeWeStartIsTheOnlyOneListed() {
env("SOLR_HEAP", "512m");
run("solr start");
run("solr create -c " + collName + " --shards 2")
.assertContains("2 shard(s)");
run("solr status --port ${SOLR_PORT}")
.assertContains("running on port");
run("solr stop --all");
run("solr status").assertContains("No Solr nodes are running");
}
{code}
Variable references in a command line are resolved by the harness, not by the
shell — which is what makes one line work under both, since bash expands
{{$NAME}} and cmd.exe does not, while cmd.exe wants {{%NAME%}} and bash does
not.
{{run()}} asserts that the command succeeded, so the happy path carries no
status boilerplate; a command expected to fail uses {{{}runFail(){}}}, which
asserts a non-zero status and hands back the result to assert on what it
printed.
h3. One source, both shells
Each test runs under bash on Linux and macOS, and under cmd.exe on Windows. A
platform-specific construct — a Unix-only tool, a pipe, command substitution —
is rejected at the call site with the fluent method that replaces it named in
the message, rather than silently passing on one platform. A test that
genuinely is one shell's carries {{{}@EnabledOnShell{}}}.
Suites run in parallel forks: each test class gets its own {{{}SOLR_HOME{}}},
its own {{SOLR_PID_DIR}} and its own block of ports.
A failure prints the command transcript — every command, its exit code and its
output — and one {{FAILED:}} line.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
