I'm wondering if anyone has tried using value classes in RDDs? My use case is that I have a number of RDDs containing strings, e.g.
val r1: RDD[(String, (String, Int)] = ... val r2: RDD[(String, (String, Int)] = ... and it might be clearer if I wrote case class ID(val id: String) extends AnyVal case class Name(val id: String) extends AnyVal val r1: RDD[(ID, (Name, Int)] = ... val r2: RDD[(Name, (ID, Int)] = ... This seems like a pretty typical use case for value classes, but I haven't noticed anyone talking about it. Although, I think you'd have to read through all of the Spark code paths to know whether allocation is required (http://docs.scala-lang.org/overviews/core/value-classes.html), so some comparative performance testing would be called for. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Anyone-using-value-classes-in-RDDs-tp4464.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
