>From my limited knowledge, all classes involved with the RDD operations
should be extending Serializable if you want Java serialization(default).
However, if you want Kryo serialization, you can
use conf.set("spark.serializer","org.apache.spark.serializer.KryoSerializer");
If you also want to perform custom serialization, as in you want some
variables to be set diferently/computed etc while deserialization, you
would create a custom registrator, register your classes with it and
call conf.set("spark.kryo.registrator","mypkg.MyKryoRegistrator");
If I am missing something, please feel free to correct me.
Best Regards,
Sonal
Nube Technologies <http://www.nubetech.co>
<http://in.linkedin.com/in/sonalgoyal>
On Sat, Mar 29, 2014 at 1:40 AM, anny9699 <[email protected]> wrote:
> Thanks a lot Ognen!
>
> It's not a fancy class that I wrote, and now I realized I neither extends
> Serializable or register with Kyro and that's why it is not working.
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Do-all-classes-involving-RDD-operation-need-to-be-registered-tp3439p3446.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
