On Thu, Aug 4, 2016 at 11:56 PM, luismattor <luismat...@gmail.com> wrote:
> How can I set the timestamp column to be NOT nullable? Hi, Given [1] it's not possible without defining your own Encoder for Dataset (that you use implicitly). It'd be something as follows: implicit def myEncoder: Encoder[MyProduct] = ??? spark.createDataset(Seq(MyProduct(new Timestamp(0), 10))) I don't know how to create the Encoder though (lack of skills). You'd need to use Encoders.product[MyProduct] as a guideline. That might help - http://stackoverflow.com/questions/36648128/how-to-store-custom-objects-in-a-dataset-in-spark-1-6. [1] https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala#L672 --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
