[
https://issues.apache.org/jira/browse/HDDS-16559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Soldatov updated HDDS-16559:
-----------------------------------
Parent: HDDS-10338
Issue Type: Sub-task (was: Bug)
> Datanode ReadBlock fails with RESOURCE_EXHAUSTED when the client response
> size exceeds the gRPC message limit
> -------------------------------------------------------------------------------------------------------------
>
> Key: HDDS-16559
> URL: https://issues.apache.org/jira/browse/HDDS-16559
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: Ozone Datanode
> Affects Versions: 2.2.1
> Reporter: Sergey Soldatov
> Assignee: Sergey Soldatov
> Priority: Major
>
> The streaming ReadBlock request carries a client-chosen responseDataSize,
> taken from ozone.client.stream.read.response-data-size. The datanode
> (KeyValueHandler.readBlockImpl) allocates the read buffer with that size and
> fills it from disk before sending the first response.
>
>
> OzoneClientConfig only rejects a zero or negative value. Any value above
> the client's gRPC inbound message limit, OZONE_SCM_CHUNK_MAX_SIZE (32 MiB),
> makes the datanode allocate that much heap per stream and send a response the
> client cannot accept. The read then fails with:
>
> RESOURCE_EXHAUSTED: gRPC message exceeds maximum size 33554432: 41958428
> Reproduce: set ozone.client.stream.read.response-data-size to 64 MiB, enable
> streaming reads, and read a 40 MiB key. Without the cap the datanode sends
> the whole block in one message and the client fails the read.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
