Requirements of objects stored in RDDs

2014-05-14 Thread Soren Macbeth
Hi, What are the requirements of objects that are stored in RDDs? I'm still struggling with an exception I've already posted about several times. My questions are: 1) What interfaces are objects stored in RDDs expected to implement, if any? 2) Are collections (be they scala, java or otherwise) h

Re: Requirements of objects stored in RDDs

2014-05-12 Thread Andrew Ash
An RDD can hold objects of any type. If you generally think of it as a distributed Collection, then you won't ever be that far off. As far as serialization, the contents of an RDD must be serializable. There are two serialization libraries you can use with Spark: normal Java serialization or Kry