Ayush Saxena created HDFS-14440:
-----------------------------------
Summary: RBF: Optimize the file write process in case of multiple
destinations.
Key: HDFS-14440
URL: https://issues.apache.org/jira/browse/HDFS-14440
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Ayush Saxena
Assignee: Ayush Saxena
In case of multiple destinations, We need to check if the file already exists
in one of the subclusters for which we use the existing getBlockLocation() API
which is by default a sequential Call,
In an ideal scenario where the file needs to be created each subcluster shall
be checked sequentially, this can be done concurrently to save time.
In another case where the file is found and if the last block is null, we need
to do getFileInfo to all the locations to get the location where the file
exists. This also can be prevented by use of ConcurrentCall since we shall be
having the remoteLocation to where the getBlockLocation returned a non null
entry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
