Hey Michael,
Thanks for the clarification. I was actually assuming the query would fail.
Ok, so this means I will have to do the validation in an RDD transformation
feeding into the SchemaRDD.
On Wed, Dec 10, 2014 at 6:27 PM, Michael Armbrust
wrote:
> As the scala doc for applySchema says, "It
As the scala doc for applySchema says, "It is important to make sure that
the structure of every [[Row]] of the provided RDD matches the provided
schema. Otherwise, there will be runtime exceptions." We don't check as
doing runtime reflection on all of the data would be very expensive. You
will o
