The proposal makes sense to me. If we are not going to make interval type ANSI-compliant in this release, we should not expose it widely.
Thanks for driving it, Kent! On Fri, Jan 17, 2020 at 10:52 AM Dr. Kent Yao <yaooq...@qq.com> wrote: > Following ANSI might be a good option but also a serious user behavior > change > to introduce two different interval types, so I also agree with Reynold to > follow what we have done since version 1.5.0, just like Snowflake and > Redshift. > > Perhaps, we can make some efforts for the current interval type to make it > more future-proofing. e.g. > 1. add unstable annotation to the CalendarInterval class. People already > use > it as UDF inputs so it’s better to make it clear it’s unstable. > 2. Add a schema checker to prohibit create v2 custom catalog table with > intervals, as same as what we do for the builtin catalog > 3. Add a schema checker for DataFrameWriterV2 too > 4. Make the interval type incomparable as version 2.4 for disambiguation of > comparison between year-month and day-time fields > 5. The 3.0 newly added to_csv should not support output intervals as same > as > using CSV file format > 6. The function to_json should not allow using interval as a key field as > same as the value field and JSON datasource, with a legacy config to > restore. > 7. Revert interval ISO/ANSI SQL Standard output since we decide not to > follow ANSI, so there is no round trip. > > Bests, > > Kent > > > > > -- > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
