Hi Marcelo,
Thanks for looking into it. I have opened a jira for this:
https://issues.apache.org/jira/browse/SPARK-21494
And yes, it works fine with internal shuffle service. But for our system we
have external shuffle/dynamic allocation configured by default. We wanted
to try switching from the
Also, things seem to work with all your settings if you disable use of
the shuffle service (which also means no dynamic allocation), if that
helps you make progress in what you wanted to do.
On Thu, Jul 20, 2017 at 4:25 PM, Marcelo Vanzin wrote:
> Hmm... I tried this with the new shuffle service
Hmm... I tried this with the new shuffle service (I generally have an
old one running) and also see failures. I also noticed some odd things
in your logs that I'm also seeing in mine, but it's better to track
these in a bug instead of e-mail.
Please file a bug and attach your logs there, I'll take
Hmm... that's not enough info and logs are intentionally kept silent
to avoid flooding, but if you enable DEBUG level logging for
org.apache.spark.network.crypto in both YARN and the Spark app, that
might provide more info.
On Wed, Jul 19, 2017 at 2:58 PM, Udit Mehrotra
wrote:
> So I added these
So I added these settings in yarn-site.xml as well. Now I get a completely
different error, but atleast it seems like it is using the crypto library:
ExecutorLostFailure (executor 1 exited caused by one of the running tasks)
Reason: Unable to create executor due to Unable to register with external
So i am using the standard documentation here for configuring external
shuffle service:
https://spark.apache.org/docs/latest/running-on-yarn.html#
configuring-the-external-shuffle-service
I can see on the cluster that following jars are present as well:
/usr/lib/spark/yarn/spark-2.2.0-yarn-shuffl
On Wed, Jul 19, 2017 at 1:10 PM, Udit Mehrotra
wrote:
> Is there any additional configuration I need for external shuffle besides
> setting the following:
> spark.network.crypto.enabled true
> spark.network.crypto.saslFallback false
> spark.authenticate true
Have you set these optio
Well, how did you install the Spark shuffle service on YARN? It's not
part of YARN.
If you really have the Spark 2.2 shuffle service jar deployed in your
YARN service, then perhaps you didn't configure it correctly to use
the new auth mechanism.
On Wed, Jul 19, 2017 at 12:47 PM, Udit Mehrotra
wr
Sorry about that. Will keep the list in my replies.
So, just to clarify I am not using an older version of sparks shuffle
service. This is a brand new cluster with just Spark 2.2.0 installed
alongside hadoop 2.7.3. Could there be anything else I am missing, or I can
try differently ?
Thanks !
Please include the list on your replies, so others can benefit from
the discussion too.
On Wed, Jul 19, 2017 at 11:43 AM, Udit Mehrotra
wrote:
> Hi Marcelo,
>
> Thanks a lot for confirming that. Can you explain what you mean by upgrading
> the version of shuffle service ? Wont it automatically us
Hi Spark Dev’s,
I am trying out the new Spark’s internal authentication mechanism based off
AES encryption, https://issues.apache.org/jira/browse/SPARK-19139 which has
come up in Spark 2.2.0.
I set the following properties in my spark-defaults:
spark.network.crypto.enabled true
spark.network.cryp
On Wed, Jul 19, 2017 at 11:19 AM, Udit Mehrotra
wrote:
> spark.network.crypto.saslFallback false
> spark.authenticate true
>
> This seems to work fine with internal shuffle service of Spark. However,
> when in I try it with Yarn’s external shuffle service the executors are
> unable t
12 matches
Mail list logo
