peterxcli opened a new pull request, #11412:
URL: https://github.com/apache/ozone/pull/11412
## What changes were proposed in this pull request?
HDDS-9904 adds block reads over Ratis data streams. Ratis 3.3.1 supports
read-only data streams (RATIS-1240): the client sends one request, and the
server sends the data back in a sequence of replies. This PR adds the datanode
side for ReadBlock. The client side comes in later sub-tasks of HDDS-9904.
Today a datanode cannot serve ReadBlock on a read-only stream, for two
reasons:
1. `ContainerStateMachine` does not implement `transferTo(Message,
WritableByteChannel)`, which Ratis calls to send the data of a read-only stream.
2. SCM gives the read pipeline of a closed container a random ID
(`RatisPipelineProvider#createForRead`). No Raft group has this ID, so a stream
on that pipeline fails with a group mismatch.
Changes:
- `ContainerStateMachine#transferTo` serves ReadBlock and rejects other
commands. It passes the request to the dispatcher's streaming read
(`streamDataReadOnly`, the same one the gRPC streaming read uses) and writes
each response to the stream as one reply: `[int: metadata length][the response
without its data][data]`.
- When the read is done, it closes the stream, which sends the terminal
reply.
- When the read fails (for example, the offset is past the end of the
block, or the connection breaks), it throws without closing the stream, and
Ratis sends the failure as the terminal reply.
- An error response from the dispatcher, such as `CONTAINER_NOT_FOUND`, is
sent as a reply with no data, so the client still gets the result code.
- `ClosedContainerReadResolver` is a Ratis `DataStreamApi.Resolver`
(RATIS-2603), registered in `XceiverServerRatis`. Ratis asks it before it looks
up the Raft group. It serves the read by container ID when the request is a
ReadBlock, the container is CLOSED, and the request names this datanode. For
any other request it returns null, and Ratis uses the Raft group as before. A
closed container does not change, so its read needs neither the group nor the
leader.
- A new `DatanodeVersion`, `RATIS_DATASTREAM_READ_BLOCK_SUPPORT` (5), tells
clients which datanodes can serve this read.
Block tokens are checked as in the gRPC read path: the dispatcher validates
the token when the `DispatcherContext` is null.
Each reply is built in one heap buffer, which copies the data once more.
Removing that copy is left to a later sub-task.
This is the datanode part of https://github.com/apache/ozone/pull/10743,
split out as suggested in
https://github.com/apache/ozone/pull/10743#issuecomment-5896221907. It does not
depend on HDDS-16721 (https://github.com/apache/ozone/pull/11411).
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-16722
## How was this patch tested?
- New tests in `ContainerStateMachineTests` (run by
`TestContainerStateMachineLeader` and `TestContainerStateMachineFollower`):
- each response becomes one reply with the response without its data, then
the data;
- an error response becomes a reply without data;
- a failed read or a failed write throws and leaves the stream open, and
nothing more is written after a failed write;
- commands other than ReadBlock are rejected.
- New `TestClosedContainerReadResolver`: the resolver serves ReadBlock only
for a closed container on this datanode.
- New `TestStreamRead#testRatisStreamReadBlock`: on a mini cluster, a plain
Ratis client reads a block on a read-only stream, first from the open container
through its Raft group, then, after the container is closed, through the read
pipeline from SCM, which has a random ID and no Raft group. Both reads match
the block file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
