My hunch is that you changed spark.serializer to Kryo but left
spark.closureSerializer unmodified, so it's still using Java for closure
serialization. Kryo doesn't really work as a closure serializer but
there's an open pull request to fix this:
https://github.com/apache/spark/pull/6361
On Mon, J
My program is written in Scala. I am creating a jar and submitting it using
spark-submit.
My code is on a computer in an internal network withe no internet so I
can't send it.
On Mon, Jun 22, 2015, 3:19 PM Akhil Das wrote:
> How are you submitting the application? Could you paste the code that y
How are you submitting the application? Could you paste the code that you
are running?
Thanks
Best Regards
On Mon, Jun 22, 2015 at 5:37 PM, Sean Barzilay
wrote:
> I am trying to run a function on every line of a parquet file. The
> function is in an object. When I run the program, I get an exce
I am trying to run a function on every line of a parquet file. The function
is in an object. When I run the program, I get an exception that the object
is not serializable. I read around the internet and found that I should use
Kryo Serializer. I changed the setting in the spark conf and registered
