Re: when does a Row object have a schema

2016-10-09 Thread Divya Gehlot
A value of a row can be accessed through both generic access by ordinal, which will incur boxing overhead for primitives, as well as native primitive access. An example of generic access by ordinal: import org.apache.spark.sql._ val row = Row(1, true, "a string", null) // row: Row = [1,true,a

when does a Row object have a schema

2016-10-09 Thread Koert Kuipers
the Spark-SQL Row trait has a schema that by default is null. when the schema is null operations that rely on fieldIndex such as getAs[T](fieldName: String): T do not work. i noticed that when i convert a DataFrame to Rdd[Row] that the Row objects do have schemas. can i rely on this? when can i b