Re: Mutiple spark contexts

2016-01-27 Thread Nicholas Chammas
There is a lengthy discussion about this on the JIRA: https://issues.apache.org/jira/browse/SPARK-2243 On Wed, Jan 27, 2016 at 1:43 PM Herman van Hövell tot Westerflier < hvanhov...@questtec.nl> wrote: > Just out of curiousity. What is the use case for having multiple active > contexts in a singl

Re: Mutiple spark contexts

2016-01-27 Thread Herman van Hövell tot Westerflier
Just out of curiousity. What is the use case for having multiple active contexts in a single JVM? Kind regards, Herman van Hövell 2016-01-27 19:41 GMT+01:00 Ashish Soni : > There is a property you need to set which is > spark.driver.allowMultipleContexts=true > > Ashish > > On Wed, Jan 27, 2016

Re: Mutiple spark contexts

2016-01-27 Thread Reynold Xin
There are no major obstacles, just a million tiny obstacles that would take forever to fix. On Wed, Jan 27, 2016 at 10:39 AM, Jakob Odersky wrote: > A while ago, I remember reading that multiple active Spark contexts > per JVM was a possible future enhancement. > I was wondering if this is stil

Re: Mutiple spark contexts

2016-01-27 Thread Ashish Soni
There is a property you need to set which is spark.driver.allowMultipleContexts=true Ashish On Wed, Jan 27, 2016 at 1:39 PM, Jakob Odersky wrote: > A while ago, I remember reading that multiple active Spark contexts > per JVM was a possible future enhancement. > I was wondering if this is still

Mutiple spark contexts

2016-01-27 Thread Jakob Odersky
A while ago, I remember reading that multiple active Spark contexts per JVM was a possible future enhancement. I was wondering if this is still on the roadmap, what the major obstacles are and if I can be of any help in adding this feature? regards, --Jakob ---