Hi everyone,
I would like to start a discussion about FLIP-517: Better Handling of
Dynamic Table Primitives with PTFs [1].
In the past months, I have spent a significant amount of time with SQL
semantics and the SQL standard around PTFs, when designing and
implementing FLIP-440 [2]. For those of you that have not taken a look
into the standard, the concept of Polymorphic Table Functions (PTF)
enables syntax for implementing custom SQL operators. In my opinion,
they are kind of a revolution in the SQL language. PTFs can take scalar
values, tables, models (in Flink), and column lists as arguments. With
these primitives, we can further evolve shortcomings in the Flink SQL
language by leveraging syntax and semantics.
I would like introduce a couple of built-in PTFs with the goal to make
the handling of dynamic tables easier for users. Once users understand
how a PTF works, they can easily select from a list of functions to
approach a table for snapshots, changelogs, or searching.
The FLIP proposes:
SNAPSHOT()
SEARCH_KEY()
TO_CHANGELOG()
FROM_CHANGELOG()
I'm aware that this is a delicate topic, and might lead to controversial
discussions. I hope with concise naming and syntax the benefit over the
existing syntax becomes clear.
There are more useful PTFs to come, but those are the ones that I
currently see as the most fundamental ones to tell a round story around
Flink SQL.
Looking forward to your feedback.
Thanks,
Timo
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-517%3A+Better+Handling+of+Dynamic+Table+Primitives+with+PTFs
[2]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=298781093
