Thank you for your opinions. I updated the FLIP with results of the discussion. Let me know if you have further concerns.
Best,
Dawid
On 05/03/2020 07:46, Jark Wu wrote:
> Hi Dawid,
>
>> INHERITS creates a new table with a "link" to the original table.
> Yes, INHERITS is a "link" to the original table in PostgreSQL.
> But INHERITS is not SQL standard, I think it's fine for vendors to define
> theire semantics.
>
>> Standard also allows declaring the clause after the schema part. We can
> also do it.
> Is that true? I didn't find it in SQL standard. If this is true, I prefer
> to put LIKE after the schema part.
>
> ====================================
>
> Hi Jingsong,
>
> The concern you mentioned in (2) is exactly my concern too. That's why I
> suggested INHERITS, or put LIKE after schema part.
>
> Best,
> Jark
>
> On Thu, 5 Mar 2020 at 12:05, Jingsong Li <jingsongl...@gmail.com> wrote:
>
>> Thanks Dawid for starting this discussion.
>>
>> I like the "LIKE".
>>
>> 1.For "INHERITS", I think this is a good feature too, yes, ALTER TABLE will
>> propagate any changes in column data definitions and check constraints down
>> the inheritance hierarchy. A inherits B, A and B share every things, they
>> have the same kafka topic. If modify schema of B, this means underlying
>> kafka topic schema changed, so I think it is good to modify A too. If this
>> for "ConfluentSchemaRegistryCatalog" mention by Jark, I think sometimes
>> this is just we want.
>> But "LIKE" also very useful for many cases.
>>
>> 2.For LIKE statement in schema, I know two kinds of like syntax, one is
>> MySQL/hive/sqlserver, the other is PostgreSQL. I prefer former:
>> - In the FLIP, there is "OVERWRITING OPTIONS", this will overwrite
>> properties in "with"? This looks weird, because "LIKE" is in schema, but it
>> can affect outside properties.
>>
>> Best,
>> Jingsong Lee
>>
>> On Wed, Mar 4, 2020 at 2:05 PM Dawid Wysakowicz <dwysakow...@apache.org>
>> wrote:
>>
>>> Hi Jark,
>>> I did investigate the INHERITS clause, but it has a semantic that in my
>>> opinion we definitely don't want to support. INHERITS creates a new table
>>> with a "link" to the original table. Therefore if you e.g change the
>> schema
>>> of the original table it's also reflected in the child table. It's also
>>> possible for tables like A inherits B query them like Select * from only
>> A,
>>> by default it returns results from both tables. I am pretty sure it's not
>>> what we're looking for.
>>>
>>> PostgreSQL implements both the LIKE clause and INHERITS. I am open for
>>> discussion if we should support multiple LIKE statements or not. Standard
>>> also allows declaring the clause after the schema part. We can also do
>> it.
>>> Nevertheless I think including multiple tables might be useful, e.g. when
>>> you want to union two tables and output to the same Kafka cluster and
>> just
>>> change the target topic. I know it's not a very common use case but it's
>>> not a big effort to support it.
>>>
>>> Let me know what you think.
>>>
>>> Best,
>>> Dawid
>>>
>>> On Wed, 4 Mar 2020, 04:55 Jark Wu, <imj...@gmail.com> wrote:
>>>
>>>> Hi Dawid,
>>>>
>>>> Thanks for starting this discussion. I like the idea.
>>>> Once we support more intergrated catalogs,
>>>> e.g. ConfluentSchemaRegistryCatalog, this problem will be more urgent.
>>>> Because it's very common to adjust existing tables in catalog slightly.
>>>>
>>>> My initial thought was introducing INHERITS keyword, which is also
>>>> supported in PostgreSQL [1].
>>>> This is also similar to the functionality of Hive CREATE TABLE LIKE
>> [2].
>>>> CREATE TEMPORARY TABLE MyTable (WATERMARK FOR ts) INHERITS
>>>> cat.db.KafkoTopic
>>>> CREATE TEMPORARY TABLE MyTable (WATERMARK FOR ts) INHERITS
>>>> cat.db.KafkoTopic WITH ('k' = 'v')
>>>>
>>>> The INHERITS can inherit an existing table with all columns, watermark,
>>> and
>>>> properties, but the properties and watermark and be overwrited
>>> explicitly.
>>>> The reason I prefer INHERITS rather than LIKE is the keyword position.
>> We
>>>> are copying an existing table definition including the properties.
>>>> However, LIKE appears in the schema part, it sounds like copying
>>> properties
>>>> into schema part of DDL.
>>>>
>>>> Besides of that, I'm not sure whether the use case stands "merging two
>>>> tables into a single one with a different connector".
>>>> From my understanding, most use cases are just slightly adjusting on an
>>>> existing catalog table with new properties or watermarks.
>>>> Do we really need to merge two table definitions into a single one? For
>>>> example, is it possible to merge a Kafka table definition and
>>>> a Filesystem table definition into a new Kafka table, and the new Kafka
>>>> table exactly matches the underlying physical data format?
>>>>
>>>> Best,
>>>> Jark
>>>>
>>>> [1]: https://www.postgresql.org/docs/9.5/sql-createtable.html
>>>> [2]:
>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-CreateTableLike
>>>>
>>>> On Tue, 3 Mar 2020 at 21:12, Dawid Wysakowicz <dwysakow...@apache.org>
>>>> wrote:
>>>>
>>>>> Hi devs,
>>>>>
>>>>> I wanted to bring another improvement proposal up for a discussion.
>>> Often
>>>>> users need to adjust existing tables slightly. This is especially
>>> useful
>>>>> when users need to enhance a table created from an external tool
>> (e.g.
>>>>> HIVE) with Flink's specific information such as e.g watermarks. It
>> can
>>>> also
>>>>> be a useful tool for ETL processes, e.g. merging two tables into a
>>> single
>>>>> one with a different connector. My suggestion would be to support an
>>>>> optional *Feature T171, “LIKE clause in table definition” *of SQL
>>>>> standard 2008.
>>>>>
>>>>> You can see the description of the proposal here:
>>>>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-110%3A+Support+LIKE+clause+in+CREATE+TABLE
>>>>> Looking forward for your comments.
>>>>>
>>>>> Best,
>>>>>
>>>>> Dawid
>>>>>
>>
>> --
>> Best, Jingsong Lee
>>
signature.asc
Description: OpenPGP digital signature
