Zheng Shao created HDFS-11280:
---------------------------------
Summary: Allow WebHDFSClient to reuse HTTP connections (HTTP
Keep-Alive)
Key: HDFS-11280
URL: https://issues.apache.org/jira/browse/HDFS-11280
Project: Hadoop HDFS
Issue Type: Improvement
Components: hdfs
Affects Versions: 3.0.0-alpha1, 2.6.5, 2.7.3
Reporter: Zheng Shao
Priority: Minor
WebHDFSClient calls "conn.disconnect()", which disconnects from the NameNode.
When we use webhdfs as the source in distcp, this used up all ephemeral ports
since all closed connections continue to occupy the port with TIME_WAIT status
for some time.
According to tinyurl.com/java7-http-keepalive, we should call
conn.getInputStream().close() instead to make sure the connection is kept
alive. This will get rid of the ephemeral port problem.
Reference:
2.7 and below:
https://github.com/apache/hadoop/blob/branch-2.6/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java#L743
2.8 and above:
https://github.com/apache/hadoop/blob/branch-2.8/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java#L898
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
