WangJianfei created SPARK-17535:
-----------------------------------
Summary: Performance Improvement of Signleton pattern in
SparkContext
Key: SPARK-17535
URL: https://issues.apache.org/jira/browse/SPARK-17535
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 2.0.0
Reporter: WangJianfei
I think the singleton pattern of SparkContext is inefficient if there are many
request to get the SparkContext,
So we can write the singleton pattern as below:
```
// the current version
def getOrCreate(): SparkContext = {
SPARK_CONTEXT_CONSTRUCTOR_LOCK.synchronized {
if (activeContext.get() == null) {
setActiveContext(new SparkContext(), allowMultipleContexts = false)
}
activeContext.get()
}
}
// by myself
def getOrCreate(): SparkContext = {
if (activeContext.get() == null) {
SPARK_CONTEXT_CONSTRUCTOR_LOCK.synchronized {
if(activeContext.get == null) {
@volatile val sparkContext = new SparkContext()
setActiveContext(sparkContext, allowMultipleContexts = false)
}
}
activeContext.get()
}
}
```
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
