Re: Monitor Job on Yarn

2016-01-04 Thread Marcelo Vanzin
You should be looking at the YARN RM web ui to monitor YARN applications; that will have a link to the Spark application's UI, along with other YARN-related information. Also, if you run the app in client mode, it might be easier to debug it until you know it's running properly (since you'll see d

Re: Monitor Job on Yarn

2016-01-04 Thread Daniel Valdivia
I see, I guess I should have set the historyServer. Strangely enough peeking in the yarn seems like nothing is "happening", it list a single application running with 0% progress but each node has 0 running containers which confuses me to wether anything is actually happening Should I restart t

Re: Monitor Job on Yarn

2016-01-04 Thread Ted Yu
Please look at history server related content under: https://spark.apache.org/docs/latest/running-on-yarn.html Note spark.yarn.historyServer.address FYI On Mon, Jan 4, 2016 at 2:49 PM, Daniel Valdivia wrote: > Hello everyone, happy new year, > > I submitted an app to yarn, however I'm unable to

Monitor Job on Yarn

2016-01-04 Thread Daniel Valdivia
Hello everyone, happy new year, I submitted an app to yarn, however I'm unable to monitor it's progress on the driver node, not in :8080 or :4040 as documented, when submitting to the standalone mode I could monitor however seems liek its not the case right now. I submitted my app this way: s