zeroshade commented on code in PR #496:
URL: https://github.com/apache/parquet-format/pull/496#discussion_r2158051999
##########
src/main/thrift/parquet.thrift:
##########
@@ -461,6 +461,29 @@ struct GeographyType {
2: optional EdgeInterpolationAlgorithm algorithm;
}
+/**
+ * Year-Month Interval logical type annotation
+ *
+ * The data is stored as an 4 byte signed integer which represents the number
+ * of months associated with the time interval. The value can be negative to
+ * indicate a backward duration.
+ *
+ * Allowed for physical type: INT32
+ */
+struct IntervalYearMonthType {
+}
+
+/**
+ * Month-Day Interval logical type annotation
+ *
+ * The data is stored as a 16-byte signed value, which represents the number
Review Comment:
> SQL standards ....
There's plenty in Parquet that isn't identical to SQL standards. My point is
basically, do we really need to *match* SQL standards in Parquet?
For example, looking at the
[CalendarInterval](https://github.com/apache/spark/blob/master/common/unsafe/src/main/java/org/apache/spark/unsafe/types/CalendarInterval.java)
spark looks like they *almost* match the arrow MonthDayNano (they use micros
instead of nanos).
Even using an int32 of months for the YearToMonth interval doesn't match the
SQL standard exactly, but spark and many other columnar compute engines use
int32 instead of int64.
I just think that there's more benefit to prioritizing format compatibility
and reducing/avoiding conversions then to necessarily match the SQL standard.
That's my personal opinion at least. Thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
