Re: Issue submitting spark job to yarn

2014-07-25 Thread Ron Gonzalez
Turns out that it had the spark assembly in the target/dependency dir so resource localization failed because spark was writing both the assembly from the dependency dir along with the main resource localization of the assembly as part of the main code line. Yarn doesn't like it if the file is o

Issue submitting spark job to yarn

2014-07-25 Thread Ron Gonzalez
Folks,   I've been able to submit simple jobs to yarn thus far. However, when I did something more complicated that added 194 dependency jars using --addJars, the job fails in YARN with no logs. What ends up happening is that no container logs get created (app master or executor). If I add just