Re: Trying to make spark-jobserver work with yarn

Thanks Akhil, that will help a lot ! It turned out that spark-jobserver does not work in "development mode" but if you deploy a server it works (looks like the dependencies when running jobserver from sbt are not right) On Thu, Jan 1, 2015 at 5:22 AM, Akhil Das wrote: > Hi Fernando, > > He

Re: Trying to make spark-jobserver work with yarn

Hi Fernando, Here's a simple log parser/analyser written in scala (you can run it without spark-shell/submit). https://github.com/sigmoidanalytics/Test Basically to run a spark job without spark-submit or shell you need a build file

Re: Trying to make spark-jobserver work with yarn

Before jumping into a sea of dependencies and bash files: Does anyone have an example of how to run a spark job without using spark-submit or shell ? On Tue, Dec 30, 2014 at 3:23 PM, Fernando O. wrote: > Hi all, > I'm investigating spark for a new project and I'm trying to use > spark-jobser

Trying to make spark-jobserver work with yarn

Hi all, I'm investigating spark for a new project and I'm trying to use spark-jobserver because... I need to reuse and share RDDs and from what I read in the forum that's the "standard" :D Turns out that spark-jobserver doesn't seem to work on yarn, or at least it does not on 1.1.1 My config