Andrey Yarovoy created HDDS-17719:
-------------------------------------
Summary: Avoid Token.encodeToUrlString() on every datanode read
request
Key: HDDS-17719
URL: https://issues.apache.org/jira/browse/HDDS-17719
Project: Apache Ozone
Issue Type: Improvement
Components: Ozone Client
Reporter: Andrey Yarovoy
Assignee: Andrey Yarovoy
*Description*
Every datanode command built by `ContainerProtocolCalls` re-encodes the block
token:
{code:java}
if (token != null) {
builder.setEncodedToken(token.encodeToUrlString());
}
{code}
This applies to `getBlock`, `getBlockFromDatanode`, `readChunk`, `listBlock`,
`getCommittedBlockLength`, `finalizeBlock`, `writeSmallFile`,
`readSmallFile` and `buildReadBlockCommandProto` (9 sites), and to
`BlockInputStream`'s short-circuit GetBlock.
`Token.encodeToUrlString` serializes the token `Writable` into a
`DataOutputBuffer` and Base64-URL-encodes it (commons-codec): an allocation, a
copy
and an encode per request. The token for a block does not change between
requests until it is refreshed.
The write path already avoids this: `BlockOutputStream` and
`BlockDataStreamOutput` compute `tokenString` once in the constructor.
*Evidence (HBase YCSB on EC, RegionServer async-profiler JFR, 6×60s per arm)*
`Token.encodeToUrlString` costs **0.3% (stream read ON) and 0.5% (OFF)** of RS
CPU, about 3% of the classic read path.
`org/apache/commons/codec/binary/Base64.encode` appears among the RS self
frames.
*Proposed fix*
1. Add overloads of the read-side `ContainerProtocolCalls` methods
(`readChunk`, `getBlock`, `buildReadBlockCommandProto`) that take the
already-encoded token `String`. Keep the `Token` overloads delegating, so other
callers are unaffected.
2. In `BlockInputStream`, `ChunkInputStream`/`ChunkReader` and
`StreamBlockInputStream`, keep the encoded string alongside the `tokenRef` and
recompute it only when the token instance changes. That happens on
`refreshBlockInfo` / token-expiry retry, when the stream gets a new token
from OM. A small holder pairing the token with its encoding (compared by
identity) keeps this to one field.
{code:java}
private static final class EncodedToken {
private final Token<OzoneBlockTokenIdentifier> token;
private final String encoded;
...
static EncodedToken of(Token<OzoneBlockTokenIdentifier> t) throws IOException {
return t == null ? null : new EncodedToken(t, t.encodeToUrlString());
}
}
{code}
*Compatibility / side effects*
None. The same bytes go on the wire; this only caches the string per token
instance. No public API removed.
*Tests*
- Unit: across N reads of one block, `encodeToUrlString` is called once per
token instance (Mockito spy). After a token refresh (simulated
`BLOCK_TOKEN_VERIFICATION_FAILED` retry), the new token's encoding is sent.
- Existing secure read tests: `TestSecureOzoneRpcClient`, block-token expiry
tests.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
