Hi Dawid,
Thanks for starting this discussion. I like the idea.
Once we support more intergrated catalogs,
e.g. ConfluentSchemaRegistryCatalog, this problem will be more urgent.
Because it's very common to adjust existing tables in catalog slightly.
My initial thought was introducing INHERITS keyword, which is also
supported in PostgreSQL [1].
This is also similar to the functionality of Hive CREATE TABLE LIKE [2].
CREATE TEMPORARY TABLE MyTable (WATERMARK FOR ts) INHERITS cat.db.KafkoTopic
CREATE TEMPORARY TABLE MyTable (WATERMARK FOR ts) INHERITS
cat.db.KafkoTopic WITH ('k' = 'v')
The INHERITS can inherit an existing table with all columns, watermark, and
properties, but the properties and watermark and be overwrited explicitly.
The reason I prefer INHERITS rather than LIKE is the keyword position. We
are copying an existing table definition including the properties.
However, LIKE appears in the schema part, it sounds like copying properties
into schema part of DDL.
Besides of that, I'm not sure whether the use case stands "merging two
tables into a single one with a different connector".
>From my understanding, most use cases are just slightly adjusting on an
existing catalog table with new properties or watermarks.
Do we really need to merge two table definitions into a single one? For
example, is it possible to merge a Kafka table definition and
a Filesystem table definition into a new Kafka table, and the new Kafka
table exactly matches the underlying physical data format?
Best,
Jark
[1]: https://www.postgresql.org/docs/9.5/sql-createtable.html
[2]:
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-CreateTableLike
On Tue, 3 Mar 2020 at 21:12, Dawid Wysakowicz <dwysakow...@apache.org>
wrote:
> Hi devs,
>
> I wanted to bring another improvement proposal up for a discussion. Often
> users need to adjust existing tables slightly. This is especially useful
> when users need to enhance a table created from an external tool (e.g.
> HIVE) with Flink's specific information such as e.g watermarks. It can also
> be a useful tool for ETL processes, e.g. merging two tables into a single
> one with a different connector. My suggestion would be to support an
> optional *Feature T171, “LIKE clause in table definition” *of SQL
> standard 2008.
>
> You can see the description of the proposal here:
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-110%3A+Support+LIKE+clause+in+CREATE+TABLE
>
> Looking forward for your comments.
>
> Best,
>
> Dawid
>
