In spark-avro 0.1, the method AvroContext.avroFile returns a SchemaRDD, which
is deprecated in Spark 1.3.0
package com.databricks.spark
import org.apache.spark.sql.{SQLContext, SchemaRDD}
package object avro {
/**
* Adds a method, `avroFile`, to SQLContext that allows reading data stored
in Avro.
*/
implicit class AvroContext(sqlContext: SQLContext) {
def avroFile(filePath: String) =
sqlContext.baseRelationToSchemaRDD(AvroRelation(filePath)(sqlContext))
}
}
Is there a new version of spark-avro, so that AvroContext.avroFile returns a
DataFrame ?
In github, spark-avro is still in version 0.1 .
databricks/spark-avro
| |
| | | | | | | |
| databricks/spark-avrospark-avro - Integration utilities for using Spark with
Apache Avro data |
| |
| View on github.com | Preview by Yahoo |
| |
| |
Thanks in advance for your assistance !
Shing
