N shuffle service for Spark on port 7337. Authentication is not
enabled.
Appreciate all and the pointers where to look at. Thanks, problem solved.
Date: Tue, 21 Jul 2015 09:31:50 -0700
Subject: Re: The auxService:spark_shuffle does not exist
From: and...@databricks.com
To: alee...@hotmail.com
CC
Hi Andrew,
Based on your driver logs, it seems the issue is that the shuffle service
is actually not running on the NodeManagers, but your application is trying
to provide a "spark_shuffle" secret anyway. One way to verify whether the
shuffle service is actually started is to look at the NodeManag
nabled the spark_shuffle via the documentation
https://spark.apache.org/docs/1.4.1/job-scheduling.html ??
I'm testing it on Hadoop 2.4.1.
Any feedback or suggestion are appreciated, thanks.
Date: Fri, 17 Jul 2015 15:35:29 -0700
Subject: Re: The auxService:spark_shuffle does not exist
From: and...
ava:152)
> at
> org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)
>
>
> Not sure what else am I missing here or doing wrong?
>
> Appreciate any insights or feedback, thanks.
>
>
> -
or doing wrong?
Appreciate any insights or feedback, thanks.
Date: Wed, 8 Jul 2015 09:25:39 +0800
Subject: Re: The auxService:spark_shuffle does not exist
From: zjf...@gmail.com
To: rp...@njit.edu
CC: user@spark.apache.org
Did you enable the dynamic resource allocation ? You can refer to this pa
Did you enable the dynamic resource allocation ? You can refer to this page
for how to configure spark shuffle service for yarn.
https://spark.apache.org/docs/1.4.0/job-scheduling.html
On Tue, Jul 7, 2015 at 10:55 PM, roy wrote:
> we tried "--master yarn-client" with no different result.
>
>
>
we tried "--master yarn-client" with no different result.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/The-auxService-spark-shuffle-does-not-exist-tp23662p23689.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---
