Re: weird YARN errors on new Spark on Yarn cluster

2014-10-02 Thread Andrew Or
asted in the original email, btw. > > Greg > > From: Andrew Or > Date: Thursday, October 2, 2014 12:24 PM > To: Greg > Cc: "user@spark.apache.org" > Subject: Re: weird YARN errors on new Spark on Yarn cluster > > Hi Greg, > > Have you looked

Re: weird YARN errors on new Spark on Yarn cluster

2014-10-02 Thread Greg Hill
gt;> Cc: "user@spark.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>> Subject: Re: weird YARN errors on new Spark on Yarn cluster Hi Greg, Have you looked at the AM container logs? (You may already know this, but) you can get these through the RM web U

Re: weird YARN errors on new Spark on Yarn cluster

2014-10-02 Thread Andrew Or
Hi Greg, Have you looked at the AM container logs? (You may already know this, but) you can get these through the RM web UI or through: yarn logs -applicationId If an AM throws an exception then the executors may not be started properly. -Andrew 2014-10-02 9:47 GMT-07:00 Greg Hill : > I h

weird YARN errors on new Spark on Yarn cluster

2014-10-02 Thread Greg Hill
I haven't run into this until today. I spun up a fresh cluster to do some more testing, and it seems that every single executor fails because it can't connect to the driver. This is in the YARN logs: 14/10/02 16:24:11 INFO executor.CoarseGrainedExecutorBackend: Connecting to driver: akka.tcp: