Hello Spark community - Does Spark 2.0 Datasets *not support* Scala Value classes (basically "extends AnyVal" with a bunch of limitations) ?
I am trying to do something like this: case class FeatureId(value: Int) extends AnyVal val seq = Seq(FeatureId(1),FeatureId(2),FeatureId(3)) import spark.implicits._ val ds = spark.createDataset(seq) ds.count This will compile, but then it will break at runtime with a cryptic error about "cannot find int at value". If I remove the "extends AnyVal" part, then everything works. Value classes are a great performance boost / static type checking feature in Scala, but are they prohibited in Spark Datasets? Thanks!
