yandrey321 opened a new pull request, #11884:
URL: https://github.com/apache/ozone/pull/11884
## What changes were proposed in this pull request?
**Problem.** Every conversion to `ECReplicationConfig` creates a new object,
although only a few EC configs are used
in practice. `ReplicationConfig.fromProto` is called for every key,
container and pipeline read from a proto, so OM,
SCM and clients keep many equal copies alive, one per `OmKeyInfo`,
`ContainerInfo` or `Pipeline`.
`RatisReplicationConfig` and `StandaloneReplicationConfig` already return
shared instances from `getInstance`.
**Change.**
- `ECReplicationConfig` creates one shared instance, at class loading, for
each config that
`ozone.replication.allowed-configs` allows by default
(`EC/(3-2|6-3|10-4)-(512|1024|2048|4096)k`), with each codec
(RS and XOR): 24 instances.
- New `ECReplicationConfig.getInstance(String)` and
`getInstance(HddsProtos.ECReplicationConfig)` return the shared
instance for those configs. Any other config is created as before.
- For protos, the lookup reads the fields directly and matches the codec
with `equalsIgnoreCase`, so a predefined
config is returned without creating any object. The proto constructor
upper-cases the codec name and looks it up
with `EcCodec.valueOf`, which creates and hashes a new string on every
call; other configs are now built from the
matched codec, which skips that too.
- For strings, a new private helper `parse(String)` extracts the codec,
data, parity and chunk size. It first looks
the string up in an immutable map (`ImmutableMap`, built once at class
loading) from the 24 predefined configs'
usual form (`rs-6-3-1024k`, as `getReplication()` prints them) to the
shared instance, and returns that instance
without running the regex. Other strings go through the regex as before,
with the same checks and error messages, and are shared too if they turn
out to be a predefined config (for
example `RS-6-3-1024K` or `rs-6-3-1048576`). The public String constructor
uses the same helper and copies the
fields, so it skips the regex for the usual forms as well, and still
returns a new object.
- Callers that convert from a string or proto use `getInstance`:
- `ReplicationConfig.fromProto` and `parseWithoutFallback`;
- `DefaultReplicationConfig.fromProto`;
- SCM's `StorageContainerLocationProtocolServerSideTranslatorPB`;
- `ReconstructECContainersCommand.getFromProtobuf`;
- the S3 Gateway's `S3Utils`.
**Key decisions.**
- **A fixed set, not a cache.** The protos come from clients, and the server
doesn't validate them against
`ozone.replication.allowed-configs` yet (HDDS-6048), so a cache filled
from them would have to be bounded and could
be filled with unused configs. The supported configs are known in advance,
so they are created up front; nothing is
added at runtime, so there is no growth, no eviction, no lock and no race.
- **The default allowed configs, not the configured ones.**
`allowed-configs` can be changed per configuration, but
the shared instances live in a static table that doesn't depend on a
configuration. A cluster that allows other
configs still works; those configs are just not shared. Validation is
unchanged.
- **Both codecs.** The default `allowed-configs` pattern doesn't restrict
the codec, so XOR configs are included.
- **Constructors are kept** and are not deprecated: they are public API, and
more than 250 test call sites use them.
`S3StorageType`'s `new ECReplicationConfig(3, 2)` is an enum constant
created once, so it is unchanged.
Generated-by: Claude Code (Claude Opus 5.5)
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-17570
## How was this patch tested?
Unit + integration tests.
- **Benchmark:** new `ECReplicationConfigDecodeBenchmark` (tagged
`benchmark`, so not part of normal runs). It uses
only public API, so the same class ran on master and on this branch, 2
runs each, JDK 21, with
`-Djacoco.skip=true` (the JaCoCo agent that surefire attaches by default
slows instrumented code down, most of all
with several threads). It decodes protos and strings with 3 common EC
configs (rs-3-2, rs-6-3, rs-10-4 at
1024k), and with 3 configs outside the predefined set (rs-4-2-1024k,
rs-6-3-8192k, rs-12-4-1024k):
- retained heap: 1,000,000 decoded objects are kept and the heap they
retain is measured after GC;
- decode: time and allocation per call on one thread, best of 5 rounds of
1 s;
- throughput: total calls/ms with 1, 4, 8 and 16 threads, best of 5 rounds
of 1 s.
| common configs | master | this PR | |
|---|---|---|---|
| distinct `ECReplicationConfig` instances held by 1M decoded objects |
1,000,000 | 3 | |
| heap retained by 1M `ReplicationConfig.fromProto` results | 30.5 MB | 0
MB | 32 bytes per object saved |
| heap retained by 1M `ContainerInfo.fromProtobuf` results | 145.8 MB |
115.0–115.2 MB | −21% |
| `ReplicationConfig.fromProto`, ns/call | 11.1 | 6.8–6.9 | 1.6x faster |
| `ReplicationConfig.fromProto`, bytes/call | 80 | 0 | |
| `ContainerInfo.fromProtobuf`, ns/call | 53.8–54.0 | 47.6–48.2 | −11% |
| `ContainerInfo.fromProtobuf`, bytes/call | 304 | 224 | −26% |
| `new ECReplicationConfig(String)`, ns/call | 161.3–162.5 | 6.0–7.0 |
23–27x faster |
| `new ECReplicationConfig(String)`, bytes/call | 528 | 32 | |
| `fromProto` calls/ms, 1 / 4 / 8 / 16 threads | 89k / 340k / 662k /
820–835k | 146–150k / 570–581k / 1.11–1.13M / 1.41–1.48M | 1.65–1.75x |
| `ContainerInfo` calls/ms, 1 / 4 / 8 / 16 threads | 18.5k / 29–30k / 24k
/ 11k | 21–22k / 30k / 23–24k / 12–16k | 1.15x at 1 thread, same after |
| other configs | master | this PR |
|---|---|---|
| `ReplicationConfig.fromProto`, ns/call | 10.7 | 5.9–6.5 |
| `ReplicationConfig.fromProto`, bytes/call | 80 | 32 |
| `ContainerInfo.fromProtobuf`, ns/call | 54.6–54.9 | 47.2–49.4 |
| `new ECReplicationConfig(String)`, ns/call | 161.5–161.9 | 164.1–174.0 |
| `new ECReplicationConfig(String)`, bytes/call | 528 | 528–560 |
| heap retained by 1M `ContainerInfo.fromProtobuf` results | 145.1 MB |
145.2 MB |
- **Memory:** each decoded config used to keep its own 32-byte object. SCM
saves about 30 MB of heap per million
EC containers, and every other object that holds an EC config decoded
from a proto (pipelines, keys) saves 32
bytes as well.
- **Time:** decoding a common config no longer allocates, so `fromProto`
is 1.6x faster and its throughput is
1.65–1.75x higher at every thread count. Configs outside the set are not
shared, but are cheaper to decode than
before because the codec name is no longer upper-cased and hashed.
- **Strings:** the usual form of a predefined config no longer runs the
regex, so parsing it is 22x faster; the 32
bytes left are the new object that the public constructor has to return.
Other strings cost 1.5–7.5% more, for
the failed map lookup and the predefined check before the regex result
is returned; in one of the two runs they
also allocated 32 more bytes, the object the String constructor copies
its fields from, which the JIT removed in
the other run.
- **`ContainerInfo` throughput** is the same in both arms from 4 threads
on, and stops scaling after 4 threads. That is `ContainerID.valueOf`'s
shared `WeakValueCache`, which this benchmark hits with only 3 container
IDs; it is unrelated to this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
