Re: Support for uniVocity in Spark 2.x

2016-10-06 Thread Hyukjin Kwon
Yeap, there is an option to switch Apache common CSV to Univocity in external CSV library but it become univocity by default and Apache common CSV was removed during porting it into Spark 2.0. On 7 Oct 2016 2:53 a.m., "Sean Owen" wrote: > It still uses univocity, but this is an implementation de

Re: Support for uniVocity in Spark 2.x

2016-10-06 Thread Sean Owen
It still uses univocity, but this is an implementation detail, so I don't think that has amounted to supporting or not supporting it. On Thu, Oct 6, 2016 at 4:00 PM Jean Georges Perrin wrote: > > The original CSV parser from Databricks had support for uniVocity in Spark > 1.x. Can someone confir

Support for uniVocity in Spark 2.x

2016-10-06 Thread Jean Georges Perrin
The original CSV parser from Databricks had support for uniVocity in Spark 1.x. Can someone confirm it has disappeared (per: https://spark.apache.org/docs/latest/api/java/org/apache/spark/sql/streaming/DataStreamReader.html#csv(java.lang.String)