peterxcli opened a new pull request, #11741:
URL: https://github.com/apache/ozone/pull/11741
## What changes were proposed in this pull request?
Freon has no command that reads one key through a shared input stream with
positioned reads. `ocokr` opens a new stream for each read and reads the whole
key, so it cannot show what happens when many threads use the same stream.
HDDS-17399 (#11724) changes exactly that path, and its numbers so far came from
a standalone program that is not in the repository. With this command, anyone
can rerun them.
This PR adds `ozone freon ocpr` (`ozone-client-positioned-reader`). It reads
an existing key:
- Each task is one random, aligned `readFully(position, buffer)` of
`--read-size` bytes (default 64 KiB).
- `--streams S` opens S input streams on the key, and `--clients C` creates
C `OzoneClient` instances. Thread i always reads from stream (i mod S), and
stream j is opened from client (j mod C). The mapping stays the same for the
whole run, so each stream keeps the same number of threads. The command
requires 1 <= C <= S <= threads.
- `--sequential-readers K` starts K extra threads for each stream. Each one
reads the stream from start to end in steps of `--sequential-read-size` bytes
(default 1 MiB), then seeks back to the start. They run for the whole test. If
one of them fails, the command fails.
- The report has a `positioned-read` timer, a `sequential-read` timer, a
`sequential-read-bytes` counter, and a line with the sequential read throughput
in MB/s.
For example, this is the HDDS-17399 workload: 8 threads doing positioned
reads and 1 sequential reader, all on one stream.
```
ozone freon ocpr -v bench -b bench1 -k key256m -t 8 --streams 1 --clients 1 \
--read-size 65536 --sequential-readers 1 --sequential-read-size 1048576
--duration 20s
```
With more streams, the threads stop sharing the stream lock. With more
clients, they also stop sharing the per-client state, such as the
`XceiverClientManager` cache. Comparing these runs shows which of the two
limits the reads.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-17433
## How was this patch tested?
New `TestOzoneClientPositionedReader`, which runs in the shared `FreonTests`
cluster. It writes a 256 KiB key and runs `ocpr` twice: with 1 stream and 1
client, and with 2 streams and 2 clients. Each run has one sequential reader
and 100 positioned reads on 4 threads. The test checks that all 100 reads
succeed and none fail.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
