I took a closer look and, yes the files were written with Parquet v2.
For some reason Parquet v2 was set as the default, I set it back to Parquet
v1.
Thanks Michael and Ryan for the info.
Andrei.
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/Parqu
Michael is right, the delta byte array encoding is a Parquet v2 feature.
Parquet v2 isn't finished yet, though some features are in releases and
those features will be supported in future releases. In other words,
Parquet will maintain backward-compatibility for any released v2 features.
I don't r
Hi AndreiL,
Were these files written with the Parquet V2 writer? The Spark 2.1 vectorized
reader does not appear to support that format.
Michael
> On May 9, 2017, at 11:04 AM, andreiL wrote:
>
> Hi, I am getting an exception in Spark 2.1 reading parquet files where some
> columns are DELTA_B
