[
https://issues.apache.org/jira/browse/SOLR-18642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18143639#comment-18143639
]
Jan Høydahl commented on SOLR-18642:
------------------------------------
{quote}Can the description link to CATS or add as a Jira link?
{quote}
CATS is my pet name (pun intended) for the new harness added to test-framework,
so no external library in play here. I could not find one... I did consider
making CATS into its own project on github and publish it on maven-central,
then Solr could have just wrapped it in our base test class. But I think it is
a better route to use it inhouse for some time first, and then anyone who fancy
could fork it out into its own lib further down the road.
I'll clarify the description to make it clear that CATS is a *new* thing,
provided by this PR :)
> Replace BATS with CATS
> ----------------------
>
> Key: SOLR-18642
> URL: https://issues.apache.org/jira/browse/SOLR-18642
> Project: Solr
> Issue Type: Improvement
> Components: cli, test-framework
> Reporter: Jan Høydahl
> Assignee: Jan Høydahl
> Priority: Major
> Labels: pull-request-available, test-framework, windows
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Today we have BATS for testing {{bin/solr}} and realistic CLI type tests in
> bash. We have nothing similar for Windows. SOLR-17508 explored Pester, but it
> went nowhere.
> This issue proposes *CATS* — a Cross-platform Automated Testing System — a
> Java-native harness for shell testing, so such tests run under JUnit like any
> other test, in parallel, on every platform. The name follows BATS (Bash
> Automated Testing System) deliberately: same idea, one letter and one
> platform restriction fewer.
> h3. The DSL
> Pure Java with convenience methods, so a ported BATS line reads almost
> unchanged:
> {code:java}
> @Test
> public void aNodeWeStartIsTheOnlyOneListed() {
> env("SOLR_HEAP", "512m");
> run("solr start");
> run("solr create -c " + collName + " --shards 2")
> .assertContains("2 shard(s)");
> run("solr status --port ${SOLR_PORT}")
> .assertContains("running on port");
> run("solr stop --all");
> run("solr status").assertContains("No Solr nodes are running");
> }
> {code}
> Variable references in a command line are resolved by the harness, not by the
> shell — which is what makes one line work under both, since bash expands
> {{$NAME}} and cmd.exe does not, while cmd.exe wants {{%NAME%}} and bash does
> not.
> {{run()}} asserts that the command succeeded, so the happy path carries no
> status boilerplate; a command expected to fail uses {{{}runFail(){}}}, which
> asserts a non-zero status and hands back the result to assert on what it
> printed.
> h3. One source, both shells
> Each test runs under bash on Linux and macOS, and under cmd.exe on Windows. A
> platform-specific construct — a Unix-only tool, a pipe, command substitution
> — is rejected at the call site with the fluent method that replaces it named
> in the message, rather than silently passing on one platform. A test that
> genuinely is one shell's carries {{{}@EnabledOnShell{}}}.
> Suites run in parallel forks: each test class gets its own {{{}SOLR_HOME{}}},
> its own {{SOLR_PID_DIR}} and its own block of ports.
> A failure prints the command transcript — every command, its exit code and
> its output — and one {{FAILED:}} line.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
