Timo Walther created FLINK-11184:
------------------------------------
Summary: Rework TableSource and TableSink interfaces
Key: FLINK-11184
URL: https://issues.apache.org/jira/browse/FLINK-11184
Project: Flink
Issue Type: New Feature
Components: Table API & SQL
Reporter: Timo Walther
Assignee: Timo Walther
There are a couple of shortcomings with the current {{TableSource}} and
{{TableSink}} interface design. Some of the issues are covered in a [basic
design
document|https://docs.google.com/document/d/1Yaxp1UJUFW-peGLt8EIidwKIZEWrrA-pznWLuvaH39Y/edit#heading=h.41fd6rs7b3cf]
that was published a while ago.
The design document has not been updated for some time and partially overlaps
with the [current SQL DDL
discussion|http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-SQL-DDL-Design-td25006.html]
for the {{CREATE TABLE}} statement on the ML.
What needs to be solved:
- How to unify sources and sinks in regards of schema and time attributes?
- How to define watermarks, timestamp extractors or timestamp ingestion?
- How to define primary keys and partitioning keys?
- How to differentiate between update modes for tables (i.e. how to read from a
append, retraction, or update table)?
- How to express all of the above without pulling in to many dependencies on
other Flink modules if source and sink interfaces are located in
{{flink-table-spi}} package?
As of the current state of the discussion, it seems that we might extend
{{TableSchema}} to allow for returning the information above and remove current
interfaces such as {{DefinedRowtimeAttribute}} or {{DefinedFieldMapping}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
