Data Source V2 API is available in Spark 2.3. But currently there's no official library using data source v2 api to read/write avro or orc files -- spark-avro and spark-orc are both using Data Source V1. I wonder if there's a plan in the upstream to implement those readers/writers and make them official libraries? Or maybe a Spark-Hive connector to read Avro/Orc tables using Data Source V2?
Although it's doable to implement such readers/writers on our own, but depending on the official library would make it easier to maintain the project. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
