serhiy-bzhezytskyy opened a new pull request, #80:
URL: https://github.com/apache/solr-orbit/pull/80
### Description
OpenSearch Benchmark has no operation type for waiting on merges — workloads
poll
`index-stats` until `_all.total.merges.current` reaches zero. The converter
passed that
through as `"operation-type": "index-stats"`, which no runner is registered
for, so the
converted workload aborted with `No runner available for operation type
[index-stats]`.
Solr Orbit already has the operation: `wait-for-merges`, registered in the
initial port
(`57387ed4`), and the hand-written Solr `nyc_taxis` already uses it. Its
runner does the
polling itself, so the whole idiom converts to it, and the `condition` /
`retry-until-success` keys become the runner's `retry-wait-period` and
`max-wait-seconds`.
Nothing new is introduced here — the converter just stops emitting an
operation type that
has no runner.
The condition path is what identifies the idiom, so an `index-stats`
operation whose
condition polls something else is left alone. No upstream workload has one,
so that guard
is defensive; a test pins it.
Both `nyc_taxis` references live in fragments whose Jinja2 directives keep
them from
parsing as JSON, which the converter copies verbatim, so the same conversion
is applied to
their text.
90 lines of code, 164 of tests, 1 of docs.
### Issues Resolved
Fixes #79
### Testing
- [x] New functionality includes testing
**Before / after, same converted workload, one task.** Upstream `nyc_taxis`
converted on
`main` at f0883807 and on this branch, then:
```
solr-orbit run --workload-path <converted>/nyc_taxis
--pipeline=benchmark-only \
--target-hosts=localhost:8983 --include-tasks=wait-until-merges-finish
```
| | main | this branch |
|---|---|---|
| exit code | 64 | 0 |
| | `No runner available for operation type [index-stats]` | `Using
user-provided operation type [wait-for-merges] for operation
[wait-until-merges-finish]` |
The task is `include-in-reporting: false`, so ✅ SUCCESS on its own proves
nothing about it.
Reading `~/.solr-orbit/logs/benchmark.log`, the schedule is built (`[0]
warmup iterations
and [1] iterations`) and on a copy with reporting turned on the task reports
73.207 ms
service time at a 0% error rate.
**The generated operation now matches the hand-written Solr `nyc_taxis`** in
`solr-orbit-workloads` byte for byte, key order included:
```json
{
"name": "wait-until-merges-finish",
"operation-type": "wait-for-merges",
"retry-wait-period": 2.0,
"max-wait-seconds": 600,
"include-in-reporting": false
}
```
**Reach and false positives.** Converting every upstream workload that
contains the idiom
(`clickbench`, `geonames`, `geoshape`, `http_logs`, `neural_search`,
`noaa_semantic_search`, `nyc_taxis`, `pmc`, `treccovid_semantic_search`) and
counting
`"index-stats"` in the output:
```
main 17 matches across 14 files
this branch 1 match (geonames/test_procedures/default.json)
```
All 16 that changed are operation *definitions*, and every one is this
idiom. The remaining
match is not a definition — it is a schedule reference, `"operation":
"index-stats"`, to
OpenSearch Benchmark's built-in operation of that name, which `geonames`
never defines. This
change does not address that, and `geonames/test_procedures/default.json`
comes out
byte-identical to what `main` produces.
Diffing the two trees whole: apart from the conversion timestamp in
`CONVERTED.md`, exactly
13 files differ, 16 hunks in total, and every hunk is this rewrite and
nothing else.
**Indentation.** The text path re-serialises the object, so the step is
inferred from the
matched object rather than fixed: `geoshape`'s fragments are 2-space and
`nyc_taxis`'s are
4-space, and a test covers each. Without that, the rewritten keys landed 14
spaces deep
inside `geoshape`.
`pytest tests/` → 1118 passed, 5 skipped. `ruff` clean.
**Not covered.** Two source occurrences are in nested directories under
`test_procedures/`,
which the converter does not walk, so those files never reach the output at
all. And
`geonames`' bare `"operation": "index-stats"` reference is untouched. Both
are separate
defects; I will report them separately.
---
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
