Andrey Yarovoy created HDDS-16870:
-------------------------------------
Summary: EC: Support stateless positional read for EC keys
Key: HDDS-16870
URL: https://issues.apache.org/jira/browse/HDDS-16870
Project: Apache Ozone
Issue Type: Improvement
Components: EC Client
Reporter: Andrey Yarovoy
h2. Problem
HDDS-15424 added stateless positional read to \{{MultipartInputStream}}, but its
constructor enables it only when all parts are \{{BlockInputStream}} or all are
{\{StreamBlockInputStream}}. EC parts are \{{ECBlockInputStreamProxy}}, which
does
not implement \{{PartInputStream.readPositioned}} (the default throws
{\{UnsupportedOperationException}}). EC keys therefore always use the
{\{OzoneFSInputStream}} fallback: under \{{positionedReadLock}} it saves the
position, seeks, reads and seeks back. As a result:
* preads on one EC stream are serialized, and they also serialize with the
stream's sequential reads;
* every pread moves and restores the shared stream position, with the seek
costs that come with it (see sub-task 1).
h2. HDFS behavior
{\{DFSStripedInputStream}} pread is not synchronized on the stream. It uses its
own set of block readers (\{{preaderInfos}}) and reads (and, if degraded,
decodes) only the requested range, so concurrent preads on one stream are safe
and do not disturb the sequential read position.
h2. Proposal
* Implement \{{readPositioned}} on \{{ECBlockInputStreamProxy}} without touching
the stream's sequential state:
** healthy: map the range to per-index spans and call \{{readPositioned}} on the
per-index STANDALONE block streams, which are already stateless (in parallel
with the sub-task 2 executor, or serially as a first step). Opening a
per-index stream must become safe for concurrent callers;
** degraded: per-call buffers and decoder, reading and decoding only the range
(sub-task 4);
** a failure in one pread falls back to a degraded read for that call; it does
not switch the sequential reader's mode.
* Admit EC parts in \{{MultipartInputStream}}, for example by letting each part
report whether it supports stateless positional read instead of the
{\{instanceof}} check.
TDE keys also need \{{KeyInputStream}} to implement \{{PositionedReadable}}, so
that Hadoop's \{{CryptoInputStream}} can use its stateless pread path; that is
HDDS-16396.
h2. Compatibility
Client only. No wire, configuration or data change.
h2. Testing
* Unit: \{{readPositioned}} returns the right bytes for every offset/length
class,
healthy and with 1 to parity-count missing indexes, and does not change
{\{getPos()}}.
* Unit: concurrent \{{readPositioned}} calls on one stream from several threads,
interleaved with sequential reads, all return correct data.
* Integration: extend the concurrent pread test of HDDS-16396 to EC keys.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
