Re: submit_spark_job_to_YARN

Thanks everyone for your valuable time and information. It was helpful. On Sunday, August 30, 2015, Ted Yu wrote: > This is related: > SPARK-10288 Add a rest client for Spark on Yarn > > FYI > > On Sun, Aug 30, 2015 at 12:12 PM, Dawid Wysakowicz < > wysakowicz.da...@gmail.com > > wrote: > >> Hi

Re: submit_spark_job_to_YARN

This is related: SPARK-10288 Add a rest client for Spark on Yarn FYI On Sun, Aug 30, 2015 at 12:12 PM, Dawid Wysakowicz < wysakowicz.da...@gmail.com> wrote: > Hi Ajay, > > In short story: No, there is no easy way to do that. But if you'd like to > play around this topic a good starting point wou

Re: submit_spark_job_to_YARN

Hi Ajay, In short story: No, there is no easy way to do that. But if you'd like to play around this topic a good starting point would be this blog post from sequenceIQ: blog . I heard rumors that there are some work going on to pre

Re: submit_spark_job_to_YARN

Hi David, Thanks for responding! My main intention was to submit spark Job/jar to yarn cluster from my eclipse with in the code. Is there any way that I could pass my yarn configuration somewhere in the code to submit the jar to the cluster? Thank you, Ajay On Sunday, August 30, 2015, David Mitc

Re: submit_spark_job_to_YARN

Hi Ajay, Are you trying to save to your local file system or to HDFS? // This would save to HDFS under "/user/hadoop/counter" counter.saveAsTextFile("/user/hadoop/counter"); David On Sun, Aug 30, 2015 at 11:21 AM, Ajay Chander wrote: > Hi Everyone, > > Recently we have installed spark on yar

submit_spark_job_to_YARN

Hi Everyone, Recently we have installed spark on yarn in hortonworks cluster. Now I am trying to run a wordcount program in my eclipse and I did setMaster("local") and I see the results that's as expected. Now I want to submit the same job to my yarn cluster from my eclipse. In storm basically I w