zhoushulin created HDFS-13935:
---------------------------------
Summary: The timeout used in “setReadTimeout()” should be
configurable
Key: HDFS-13935
URL: https://issues.apache.org/jira/browse/HDFS-13935
Project: Hadoop HDFS
Issue Type: Bug
Reporter: zhoushulin
The timeout used in "setReadTimeout" in "Fetcher.java" and
"JobEndNotifier.java" in MapReduce is configurable as follows:
{code:title="/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java"}
class Fetcher<K,V> extends Thread {
...
this.readTimeout =
job.getInt(MRJobConfig.SHUFFLE_READ_TIMEOUT, DEFAULT_READ_TIMEOUT);
...
private void setupShuffleConnection(String encHash) {
// put url hash into http header
connection.addRequestProperty(
SecureShuffleUtils.HTTP_HEADER_URL_HASH, encHash);
// set the read timeout
connection.setReadTimeout(readTimeout);
// put shuffle version into http header
connection.addRequestProperty(ShuffleHeader.HTTP_HEADER_NAME,
ShuffleHeader.DEFAULT_HTTP_HEADER_NAME);
connection.addRequestProperty(ShuffleHeader.HTTP_HEADER_VERSION,
ShuffleHeader.DEFAULT_HTTP_HEADER_VERSION);
}
...
}
{code}
{code:title="/org/apache/hadoop/mapreduce/v2/app/JobEndNotifier.java"}
public class JobEndNotifier implements Configurable {
...
timeout = conf.getInt(JobContext.MR_JOB_END_NOTIFICATION_TIMEOUT,
JobContext.DEFAULT_MR_JOB_END_NOTIFICATION_TIMEOUT);
...
protected boolean notifyURLOnce() {
...
conn.setReadTimeout(timeout);
...
}
...
}
{code}
The similar situations happen in "DFSZKFailoverController.java" in HDFS.
However, the "setReadTimeout()" in "FederationUtil.java",
"CredentialBasedAccessTokenProvider.java", and
"ConfRefreshTokenBasedAccessTokenProvider.java" call constants as their
argument. For example:
{code:title="/org/apache/hadoop/hdfs/server/federation/router/FederationUtil.java"}
public final class FederationUtil {
...
public static JSONArray getJmx(String beanQuery, String webAddress) {
...
URLConnection conn = jmxURL.openConnection();
conn.setConnectTimeout(5 * 1000);
...
}
...
}
{code}
I think these constant should be also configurable for different requirements
and environments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
