Yeah I got the logs and its reporting about the memory. 14/09/25 00:08:26 WARN YarnClusterScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory
Now I shifted to big cluster with more memory but here im not able to view the UI while the job is running .. I need to check the status and spark UI of job. cse-hadoop-xx:18080 doesnt have running job it only had the jobs with master as spark://cse-hadoop-xx:7077 and not yarn-cluster . Im not ableto view the following links .. http://cse-hadoop-xx:50070/ http://cse-hadoop-xx:8088/ Is this due to some security option im not able to view the UI how can change it in cloudera.? On Thu, Sep 25, 2014 at 12:04 AM, Marcelo Vanzin <van...@cloudera.com> wrote: > You need to use the command line yarn application that I mentioned > ("yarn logs"). You can't look at the logs through the UI after the app > stops. > > On Wed, Sep 24, 2014 at 11:16 AM, Raghuveer Chanda > <raghuveer.cha...@gmail.com> wrote: > > > > Thanks for the reply .. This is the error in the logs obtained from UI at > > > http://dml3:8042/node/containerlogs/container_1411578463780_0001_02_000001/chanda > > > > So now how to set the Log Server url .. > > > > Failed while trying to construct the redirect url to the log server. Log > > Server url may not be configured > > > > Container does not exist. > > > > > > > > > > On Wed, Sep 24, 2014 at 11:37 PM, Marcelo Vanzin <van...@cloudera.com> > > wrote: > >> > >> You'll need to look at the driver output to have a better idea of > >> what's going on. You can use "yarn logs --applicationId blah" after > >> your app is finished (e.g. by killing it) to look at it. > >> > >> My guess is that your cluster doesn't have enough resources available > >> to service the container request you're making. That will show up in > >> the driver as periodic messages that no containers have been allocated > >> yet. > >> > >> On Wed, Sep 24, 2014 at 10:25 AM, Raghuveer Chanda > >> <raghuveer.cha...@gmail.com> wrote: > >> > Hi, > >> > > >> > I'm new to spark and facing problem with running a job in cluster > using > >> > YARN. > >> > > >> > Initially i ran jobs using spark master as --master spark://dml2:7077 > >> > and it > >> > is running fine on 3 workers. > >> > > >> > But now im shifting to YARN, so installed YARN in cloud era on 3 node > >> > cluster and changed the master to yarn-cluster but it is not working I > >> > attached the screenshots of UI which are not progressing and just > >> > hanging > >> > on. > >> > > >> > Output on terminal : > >> > > >> > This error is repeating > >> > > >> > ./spark-submit --class "class-name" --master yarn-cluster > >> > --num-executors 3 > >> > --executor-cores 3 jar-with-dependencies.jar > >> > > >> > > >> > Do i need to configure YARN or why it is not getting all the workers > .. > >> > please help ... > >> > > >> > > >> > 14/09/24 22:44:21 INFO yarn.Client: Application report from ASM: > >> > application identifier: application_1411578463780_0001 > >> > appId: 1 > >> > clientToAMToken: null > >> > appDiagnostics: > >> > appMasterHost: dml3 > >> > appQueue: root.chanda > >> > appMasterRpcPort: 0 > >> > appStartTime: 1411578513545 > >> > yarnAppState: RUNNING > >> > distributedFinalState: UNDEFINED > >> > appTrackingUrl: > http://dml2:8088/proxy/application_1411578463780_0001/ > >> > appUser: chanda > >> > 14/09/24 22:44:22 INFO yarn.Client: Application report from ASM: > >> > application identifier: application_1411578463780_0001 > >> > appId: 1 > >> > clientToAMToken: null > >> > appDiagnostics: > >> > appMasterHost: dml3 > >> > appQueue: root.chanda > >> > appMasterRpcPort: 0 > >> > appStartTime: 1411578513545 > >> > yarnAppState: RUNNING > >> > distributedFinalState: UNDEFINED > >> > appTrackingUrl: > http://dml2:8088/proxy/application_1411578463780_0001/ > >> > > >> > > >> > > >> > > >> > -- > >> > Regards, > >> > Raghuveer Chanda > >> > 4th year Undergraduate Student > >> > Computer Science and Engineering > >> > IIT Kharagpur > >> > > >> > > >> > --------------------------------------------------------------------- > >> > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > >> > For additional commands, e-mail: user-h...@spark.apache.org > >> > >> > >> > >> -- > >> Marcelo > > > > > > > > > > -- > > Regards, > > Raghuveer Chanda > > 4th year Undergraduate Student > > Computer Science and Engineering > > IIT Kharagpur > > > > -- > Marcelo > -- Regards, Raghuveer Chanda 4th year Undergraduate Student Computer Science and Engineering IIT Kharagpur
