yandrey321 opened a new pull request, #11878:
URL: https://github.com/apache/ozone/pull/11878
## What changes were proposed in this pull request?
**Problem.** `ReplicationConfig.parseWithoutFallback` validates every parsed
config with
`config.getObject(ReplicationConfigValidator.class)`. That creates a new
validator by reflection, injects the
`ozone.replication.allowed-configs` value and compiles its regular
expression (`@PostConstruct`), on every call.
`parse`, `getDefault` and `adjustReplication` all go through it, so the cost
is paid on hot paths such as every ofs/o3fs
`create()` (`OzoneClientUtils.resolveClientSideReplicationConfig`).
**Change.**
- New package-private `ReplicationConfigValidator.get(ConfigurationSource)`.
It reads the pattern from the given
configuration on every call, and reuses the validator it built last time
when the pattern is the same. Only a new
pattern creates (and compiles) a new validator.
- `parseWithoutFallback` uses it instead of `getObject`.
- The key and default value are now constants, also used by the `@Config`
annotation, so both read the same default.
**Key decisions.**
- The pattern is still read from the configuration each time, so changing
`ozone.replication.allowed-configs` in a
configuration object takes effect immediately, as before. Only the
reflection and regex compilation are saved.
- One cached entry, not a map: a JVM almost always uses a single pattern. If
callers with different patterns
interleave, each call still validates with a validator for its own
pattern; it just builds a new one, as before.
- No locking: the cache is a `volatile` reference to a validator that is
fully built before it is published and is
never changed after that. Two threads that race on a new pattern may both
build one, which is harmless.
- `get` is package-private because the returned validator is shared; callers
outside the package that mutate it
(`setValidationPattern`, `disableValidation`) still create their own
through `getObject`. `RpcClient` and
`OzoneManager` already create one validator at start-up, so they are
unchanged.
Generated-by: Claude Code (Claude Opus 5.5)
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-17569
## How was this patch tested?
Unit and Integration tests.
- **Benchmark:** new `ReplicationConfigParseBenchmark` (tagged `benchmark`,
so not part of normal runs). It uses
only public API, so the same class ran on master and on this branch, 2
runs each, JDK 21, with an
`OzoneConfiguration` shared by all threads, as in a `FileSystem`:
| | master | this PR | |
|---|---|---|---|
| `parse`, ns/call (1 thread) | 800–906 | 94–97 | 8.5–9x faster |
| `adjustReplication`, ns/call (1 thread) | 805–872 | 97–102 | 8.3–8.5x
faster |
| allocated bytes/call | 4,192–4,240 | 344 | 12x less |
| `parse` calls/ms, 1 thread | 1,128–1,207 | 10,053–10,478 | 8.7–8.9x |
| `parse` calls/ms, 4 threads | 2,539–2,870 | 3,020 | 1.1x |
| `parse` calls/ms, 8 / 16 threads | 1,470–1,575 | 2,939–3,125 | 1.9–2.1x |
With several threads, throughput is capped by Hadoop's
`Configuration.getProps()`, which is `synchronized`, so every
`conf.get` on the shared configuration takes its lock. Each call still
reads the configuration twice: `parse` reads
`ozone.replication` even when a replication is passed, and the validator
reads `ozone.replication.allowed-configs`. Master's single-thread cost is so
high that 4
threads still scale on it; with this change the time per call is small
enough that the lock dominates from 4
threads on. The gain under concurrency is therefore smaller than on one
thread, but it's still a gain at every
thread count, and allocation drops 12x regardless.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
