Re: Intro to using IntelliJ to debug SPARK-1.1 Apps with mvn/sbt (for beginners)

2014-11-18 Thread Chester @work
For sbt You can simplify run sbt/sbt gen-idea To generate the IntelliJ idea project module for you. You can the just open the generated project, which includes all the needed dependencies Sent from my iPhone > On Nov 18, 2014, at 8:26 PM, Chen He wrote: > > Thank you Yiming. It is helpful.

Re: Using CUDA within Spark / boosting linear algebra

2015-02-09 Thread Chester @work
Maybe you can ask prof john canny himself:-) as I invited him to give a talk at Alpine data labs in March's meetup (SF big Analytics & SF machine learning joined meetup) , 3/11. To be announced in next day or so. Chester Sent from my iPhone > On Feb 9, 2015, at 4:48 PM, "Ulanov, Alexander"

Re: Change for submitting to yarn in 1.3.1

2015-05-13 Thread Chester @work
Patrick Thanks for responding. Yes. many of are features requests not private client related. These are the things I have been working with since last year. I have trying to push the PR for these changes. If the new Launcher lib is the way to go , we will try to work with new APIs. T

Re: [Spark SQL] SQLContext getOrCreate incorrect behaviour

2015-12-21 Thread Chester @work
Jerry I thought you should not create more than one SparkContext within one Jvm, ... Chester Sent from my iPhone > On Dec 20, 2015, at 2:59 PM, Jerry Lam wrote: > > Hi Spark developers, > > I found that SQLContext.getOrCreate(sc: SparkContext) does not behave > correctly when a different

Re: is Branch-1.1 SBT build broken for yarn-alpha ?

2014-08-21 Thread Chester @work
Do we have Jenkins tests these ? Should be pretty easy to setup just to test basic build Sent from my iPhone > On Aug 21, 2014, at 6:45 AM, Mridul Muralidharan wrote: > > Weird that Patrick did not face this while creating the RC. > Essentially the yarn alpha pom.xml has not been updated prope

Re: Running Spark On Yarn without Spark-Submit

2014-08-29 Thread Chester @work
Archit We are using yarn-cluster mode , and calling spark via Client class directly from servlet server. It works fine. To establish a communication channel to give further requests, It should be possible with yarn client, but not with yarn server. Yarn client mode, spark driver i