Hi Jark,
I did investigate the INHERITS clause, but it has a semantic that in my
opinion we definitely don't want to support. INHERITS creates a new table
with a "link" to the original table. Therefore if you e.g change the schema
of the original table it's also reflected in the child table. It's also
possible for tables like A inherits B query them like Select * from only A,
by default it returns results from both tables. I am pretty sure it's not
what we're looking for.
PostgreSQL implements both the LIKE clause and INHERITS. I am open for
discussion if we should support multiple LIKE statements or not. Standard
also allows declaring the clause after the schema part. We can also do it.
Nevertheless I think including multiple tables might be useful, e.g. when
you want to union two tables and output to the same Kafka cluster and just
change the target topic. I know it's not a very common use case but it's
not a big effort to support it.
Let me know what you think.
Best,
Dawid
On Wed, 4 Mar 2020, 04:55 Jark Wu, <imj...@gmail.com> wrote:
> Hi Dawid,
>
> Thanks for starting this discussion. I like the idea.
> Once we support more intergrated catalogs,
> e.g. ConfluentSchemaRegistryCatalog, this problem will be more urgent.
> Because it's very common to adjust existing tables in catalog slightly.
>
> My initial thought was introducing INHERITS keyword, which is also
> supported in PostgreSQL [1].
> This is also similar to the functionality of Hive CREATE TABLE LIKE [2].
>
> CREATE TEMPORARY TABLE MyTable (WATERMARK FOR ts) INHERITS
> cat.db.KafkoTopic
> CREATE TEMPORARY TABLE MyTable (WATERMARK FOR ts) INHERITS
> cat.db.KafkoTopic WITH ('k' = 'v')
>
> The INHERITS can inherit an existing table with all columns, watermark, and
> properties, but the properties and watermark and be overwrited explicitly.
>
> The reason I prefer INHERITS rather than LIKE is the keyword position. We
> are copying an existing table definition including the properties.
> However, LIKE appears in the schema part, it sounds like copying properties
> into schema part of DDL.
>
> Besides of that, I'm not sure whether the use case stands "merging two
> tables into a single one with a different connector".
> From my understanding, most use cases are just slightly adjusting on an
> existing catalog table with new properties or watermarks.
> Do we really need to merge two table definitions into a single one? For
> example, is it possible to merge a Kafka table definition and
> a Filesystem table definition into a new Kafka table, and the new Kafka
> table exactly matches the underlying physical data format?
>
> Best,
> Jark
>
> [1]: https://www.postgresql.org/docs/9.5/sql-createtable.html
> [2]:
>
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-CreateTableLike
>
>
> On Tue, 3 Mar 2020 at 21:12, Dawid Wysakowicz <dwysakow...@apache.org>
> wrote:
>
> > Hi devs,
> >
> > I wanted to bring another improvement proposal up for a discussion. Often
> > users need to adjust existing tables slightly. This is especially useful
> > when users need to enhance a table created from an external tool (e.g.
> > HIVE) with Flink's specific information such as e.g watermarks. It can
> also
> > be a useful tool for ETL processes, e.g. merging two tables into a single
> > one with a different connector. My suggestion would be to support an
> > optional *Feature T171, “LIKE clause in table definition” *of SQL
> > standard 2008.
> >
> > You can see the description of the proposal here:
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-110%3A+Support+LIKE+clause+in+CREATE+TABLE
> >
> > Looking forward for your comments.
> >
> > Best,
> >
> > Dawid
> >
>
