Re: Multiple context in one Driver

2019-03-17 Thread Ido Friedman
You saved me lots of time. Thanks. Ido On Thu, Mar 14, 2019 at 6:18 PM Marcelo Vanzin wrote: > It doesn't work (except if you're extremely lucky), it will eat your > lunch and will also kick your dog. > > And it's not even going to be an option in the next version of Spark. > > On Wed, Mar 13,

Re: Multiple context in one Driver

2019-03-14 Thread Marcelo Vanzin
It doesn't work (except if you're extremely lucky), it will eat your lunch and will also kick your dog. And it's not even going to be an option in the next version of Spark. On Wed, Mar 13, 2019 at 11:38 PM Ido Friedman wrote: > > Hi, > > I am researching the use of multiple sparkcontext in one

Multiple context in one Driver

2019-03-13 Thread Ido Friedman
Hi, I am researching the use of multiple sparkcontext in one Driver - spark.driver.allowMultipleContexts I found various opinions and notes on the subject, most were against it and some said it is "work in progress" What is "official" approach on this subject? Can we use this as a significant pa