[
https://issues.apache.org/jira/browse/SPARK-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
WangJianfei updated SPARK-17535:
--------------------------------
Comment: was deleted
(was: After JDK1.5, The volatile can avoid the situation you say.We can't get
into some strange situations where the reference is available but the
synchronized init hasn't completed,because the volatile will tell the JVM to
not do instruction reordering about the init of sparkContext.)
> Performance Improvement of Signleton pattern in SparkContext
> ------------------------------------------------------------
>
> Key: SPARK-17535
> URL: https://issues.apache.org/jira/browse/SPARK-17535
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.0.0
> Reporter: WangJianfei
> Labels: easyfix, performance
>
> I think the singleton pattern of SparkContext is inefficient if there are
> many request to get the SparkContext,
> So we can write the singleton pattern as below,The second way if more
> efficient when there are many request to get the SparkContext.
> {code}
> // the current version
> def getOrCreate(): SparkContext = {
> SPARK_CONTEXT_CONSTRUCTOR_LOCK.synchronized {
> if (activeContext.get() == null) {
> setActiveContext(new SparkContext(), allowMultipleContexts = false)
> }
> activeContext.get()
> }
> }
> // by myself
> def getOrCreate(): SparkContext = {
> if (activeContext.get() == null) {
> SPARK_CONTEXT_CONSTRUCTOR_LOCK.synchronized {
> if(activeContext.get == null) {
> @volatile val sparkContext = new SparkContext()
> setActiveContext(sparkContext, allowMultipleContexts = false)
> }
> }
> activeContext.get()
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
