Currently Spark uses Parquet 1.7.0 (parquet-mr). If you meant writer version2 (parquet-format), you can specify this by manually setting as below:
sparkContext.hadoopConfiguration.set(ParquetOutputFormat.WRITER_VERSION, ParquetProperties.WriterVersion.PARQUET_2_0.toString) 2016-04-15 2:21 GMT+09:00 Younes Naguib <[email protected]>: > Hi all, > > > > When parquet 2.0 planned in Spark? > > Or is it already? > > > > > > *Younes Naguib* > > Triton Digital | 1440 Ste-Catherine W., Suite 1200 | Montreal, QC H3G 1R8 > > Tel.: +1 514 448 4037 x2688 | Tel.: +1 866 448 4037 x2688 | younes.naguib > @tritondigital.com <[email protected]> > > >
