ss77892 opened a new pull request, #11873:
URL: https://github.com/apache/ozone/pull/11873
## What changes were proposed in this pull request?
On the datanode, a streaming ReadBlock is served over one gRPC stream per
block, through one `RandomAccessFileChannel` that `GrpcXceiverService` keeps
open for the stream. Every ReadBlock request on that stream used to:
- look up the block's `BlockData` in the container DB via
`BlockManager.getBlock`,
- allocate a new heap `ByteBuffer` sized by the client-chosen response size,
and
- copy that buffer again into the response with `ByteString.copyFrom`.
For a sequential read of a large block this is one DB lookup, one allocation
and two copies per response. This PR removes the per-request allocation and the
extra copy, and caches the block metadata for the life of the stream:
1. **Reuse the read buffer.**
`RandomAccessFileChannel.getReadBuffer(capacity)` keeps one buffer per stream,
reallocates it only when a larger one is needed, and drops it on `close()`. The
buffer is now sized by the range actually being read rather than by the
response size the client asked for.
2. **Avoid the copy into the response.**
`ContainerCommandResponseBuilders.getReadBlockResponse` takes the handler's
`byteBufferToByteString` conversion, as `getReadChunkResponse` already does.
With `ozone.UnsafeByteOperations.enabled` (default `true`) the response wraps
the buffer instead of copying it. Reusing the buffer is safe because gRPC
serializes the response inside `StreamObserver.onNext`, before the next range
refills the buffer; this contract is documented on `getReadBlockResponse`.
3. **Cache `BlockData` per stream.** `RandomAccessFileChannel` caches the
`BlockData` keyed by `BlockID` (which includes the BCSID) and clears it on
`close()`.
- The cache is bypassed and refreshed whenever the requested range
reaches the cached block size, so a block that grew under an open stream is
re-resolved.
- On a cache hit the handler still checks `BlockManager.blockExists`, so
a block deleted from the block table keeps failing with `NO_SUCH_BLOCK`.
4. **Reject negative offset or length** with `INVALID_ARGUMENT` before the
cache check, closing the block file like the existing `OUT_OF_RANGE` rejection.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-16377
## How was this patch tested?
New unit tests:
- `TestRandomAccessFileChannel`
- `blockDataIsCachedPerBlockAndClearedOnClose`: cache hit per `BlockID`,
miss for another block or BCSID, cleared on close.
- `readBufferIsReusedGrownAndDroppedOnClose`: buffer reused and cleared,
reallocated only to grow, dropped on close.
- `TestKeyValueHandler`
- `testReadBlockReadBufferIsSizedByTheRequest`: a large client response
size does not size the buffer.
- `testReadBlockLengthBeyondTheBlockIsClamped`: `Long.MAX_VALUE` length,
aligned and misaligned offset.
- `testReadBlockCachedBlockDataIsReusedWithinTheBlock`: one `getBlock` for
two requests within the block.
- `testReadBlockCachedBlockDataIsRefreshedWhenTheBlockGrows`: a request
past the cached size sees the grown block.
- `testReadBlockCachedBlockDataBoundsTheReadWhenTheBlockGrows`: a request
inside the cached size does not read past it.
- `testReadBlockCachedBlockDataDoesNotOutliveTheBlock`: `NO_SUCH_BLOCK`
after the block is deleted.
- `testReadBlockErrorOnTheStreamClosesTheBlockFile`: extended with
negative offset and negative length.
Benchmark report vs master (3 node cluster using LXD containers)
[build_comparison_20261030_204023.pdf](https://github.com/user-attachments/files/34209571/build_comparison_20261030_204023.pdf)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
