Or launch the spark-shell with --conf spark.kryo.registrator=foo.bar.MyClass
2015-06-11 14:30 GMT+02:00 Igor Berman :
> Another option would be to close sc and open new context with your custom
> configuration
> On Jun 11, 2015 01:17, "bhomass" wrote:
>
>> you need to register using spark-defaul
Another option would be to close sc and open new context with your custom
configuration
On Jun 11, 2015 01:17, "bhomass" wrote:
> you need to register using spark-default.xml as explained here
>
>
> https://books.google.com/books?id=WE_GBwAAQBAJ&pg=PA239&lpg=PA239&dq=spark+shell+register+kryo+ser
you need to register using spark-default.xml as explained here
https://books.google.com/books?id=WE_GBwAAQBAJ&pg=PA239&lpg=PA239&dq=spark+shell+register+kryo+serialization&source=bl&ots=vCxgEfz1-2&sig=dHU8FY81zVoBqYIJbCFuRwyFjAw&hl=en&sa=X&ved=0CEwQ6AEwB2oVChMIn_iujpCGxgIVDZmICh3kYADW#v=onepage&q=
I can do that in my application, but I really want to know how I can do it
in spark-shell because I usually prototype in spark-shell before I put the
code into an application.
On Wed, Aug 20, 2014 at 12:47 PM, Sameer Tilak wrote:
> Hi Wang,
> Have you tried doing this in your application?
>
>
Hi Wang,Have you tried doing this in your application?
conf.set("spark.serializer",
"org.apache.spark.serializer.KryoSerializer")
conf.set("spark.kryo.registrator", "yourpackage.MyKryoRegistrator")
You then don't need to specify it via commandline.
Date: Wed, 20 Aug 2014 12:25:14 -0
