On Tue, 15 Dec 2020 at 17:39, Antoine Pitrou <anto...@python.org> wrote:
> > Le 15/12/2020 à 17:24, Joris Van den Bossche a écrit : > > > > But, most of the additional ConvertedTypes that were not present in > > parquet-format 1.0 (eg the different signed/unsigned integer types, > > timestamp, ..) were introduced in parquet-format 2.2 ( > > https://github.com/apache/parquet-format/pull/3, > > https://issues.apache.org/jira/browse/PARQUET-12) almost 7 years ago. > > Surely you mean "1.2"? > No, I actually mean 2.2. (I don't think a 1.2 version exists, at least according to the git tags) You can compare the thrift file for version 2.1 ( https://github.com/apache/parquet-format/blob/parquet-format-2.1.0/src/thrift/parquet.thrift) vs 2.2 ( https://github.com/apache/parquet-format/blob/apache-parquet-format-2.2.0/src/thrift/parquet.thrift), in which a set of ConvertedTypes where added. > > > If so, could we start defaulting to version 2.0 (but still with date page > > version 1.0), or do other parquet readers actually not yet support the > > ConvertedTypes introduced 7 years ago? > > +1 for defaulting to 2.0. > > Regards > > Antoine. >
