Hi everyone,
I would like to propose FLIP-65 that describes how we want to deal with
data types and their inference/extraction in the Table API in the
future. I have collected many comments, shortcomings, issues from users
and trainings in last years that went into the design. It completes the
work started in FLIP-37 by upgrading the type system to DataTypes and
allowing new use cases. Some key features of this FLIP are:
- Type extraction closely coupled to the SQL standard (with
precision/scale specification and complex data types)
- Simple stuff is simple, missing information can be provided with
annotations without the need to specify everything from scratch.
- Full access to the planner's type inference which allows to create
UDFs as powerful as built-in system functions
- Unification of Scala and Java API to enable the unified
TableEnvironment.registerFunction()
The design document can be found here:
https://docs.google.com/document/d/1Zf8-okGvCiTTRaTN0IqtTGrrjXYNGFJnhUQyinF4xcU/edit#
I will convert it to a wiki page after the first review comments.
Happy to hear your thoughts.
Thanks,
Timo
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-37%3A+Rework+of+the+Table+API+Type+System
