Following ANSI might be a good option but also a serious user behavior change to introduce two different interval types, so I also agree with Reynold to follow what we have done since version 1.5.0, just like Snowflake and Redshift.
Perhaps, we can make some efforts for the current interval type to make it more future-proofing. e.g. 1. add unstable annotation to the CalendarInterval class. People already use it as UDF inputs so it’s better to make it clear it’s unstable. 2. Add a schema checker to prohibit create v2 custom catalog table with intervals, as same as what we do for the builtin catalog 3. Add a schema checker for DataFrameWriterV2 too 4. Make the interval type incomparable as version 2.4 for disambiguation of comparison between year-month and day-time fields 5. The 3.0 newly added to_csv should not support output intervals as same as using CSV file format 6. The function to_json should not allow using interval as a key field as same as the value field and JSON datasource, with a legacy config to restore. 7. Revert interval ISO/ANSI SQL Standard output since we decide not to follow ANSI, so there is no round trip. Bests, Kent -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
