KenCao created HDFS-14809:
-----------------------------
Summary: Make a new BlockReader for hdfs client lib
Key: HDFS-14809
URL: https://issues.apache.org/jira/browse/HDFS-14809
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.6.0
Reporter: KenCao
as we known, the hdfs client java lib uses BlockReaderLocal for short circuit
read by default, which allocate shared memory first, and make a slot within it.
After all these steps, it will request the fds from the DataNode. However, the
slot and shared memory sturcture is only used by DataNode when uncaching
replicas, the client process can work well just with the fds asked later and it
is nearly impossible to cache replicas in product environment. The api to
release fds is called by client only with the slot given, the fds is close in
the client process finally.
so i think we can make a new BlockReader implementation which just requests the
fds, and it will reduce the rpc calls from 3(allocate shm, request fds, release
fds) to 1(request fds).
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
