Hi,
I am having this problem on spark 1.4. Do you have any ideas how to
solve it? I tried to use spark.executor.extraClassPath, but it did not help
BR,
Patcharee
On 04. mai 2015 23:47, Imran Rashid wrote:
Oh, this seems like a real pain. You should file a jira, I didn't see
an open issue --
I cherry-picked this commit into my local 1.2 branch. It fixed the problem
with setting spark.serializer, but I get a similar problem with
spark.closure.serializer
org.apache.spark.SparkException: Failed to register classes with Kryo
at
org.apache.spark.serializer.KryoSerializer.newKryo(Kry
Actually, after some digging, I did find a JIRA for it: SPARK-5470.
The fix for this has gone into master, but it isn't in 1.2.
On Mon, May 4, 2015 at 2:47 PM, Imran Rashid wrote:
> Oh, this seems like a real pain. You should file a jira, I didn't see an
> open issue -- if nothing else just to d
Oh, this seems like a real pain. You should file a jira, I didn't see an
open issue -- if nothing else just to document the issue.
As you've noted, the problem is that the serializer is created immediately
in the executors, right when the SparkEnv is created, but the other jars
aren't downloaded
