John Zhuge created HDFS-11900:
---------------------------------
Summary: HEDGED_READ_THREAD_POOL should not be static
Key: HDFS-11900
URL: https://issues.apache.org/jira/browse/HDFS-11900
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs-client
Affects Versions: 2.8.0
Reporter: John Zhuge
*Non-static* synchronized method initThreadsNumForHedgedReads can't synchronize
the access to the *static* class variable HEDGED_READ_THREAD_POOL.
{code}
private static ThreadPoolExecutor HEDGED_READ_THREAD_POOL;
...
private synchronized void initThreadsNumForHedgedReads(int num) {
{code}
2 DFS clients may update the same static variable in a race because the lock is
on each DFS client object, not on the shared DFSClient class object.
There are 2 possible fixes:
1. "Global thread pool": Change initThreadsNumForHedgedReads to static
2. "Per-client thread pool": Change HEDGED_READ_THREAD_POOL to non-static
>From the description for property {{dfs.client.hedged.read.threadpool.size}}:
{quote}
to a positive number. The threadpool size is how many threads to dedicate
to the running of these 'hedged', concurrent reads in your client.
{quote}
it seems to indicate the thread pool is per DFS client.
Let's assume we go with #1 "Global thread pool". One DFS client has the
property set to 10 in its config, while the other client has the property set
to 5 in its config, what is supposed to the size of the global thread pool? 5?
10? Or 15?
The 2nd fix seems more reasonable to me.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
