ent atomic for ctas, we propose a interface for
>> >> catalog
>> >> >> to support serializing, then we name it to `AtomicCatalog`. At least,
>> >> the
>> >> >> interface is for the atomic of ctas. But if we want to implement other
>> >>
: "Mang Zhang"
收件人: "dev"
抄送: imj...@gmail.com
发送时间: 星期一, 2022年 6 月 27日 下午 5:43:50
主题: Re:Re: Re:Re: Re: Re: Re: [DISCUSS] FLIP-218: Support SELECT clause in
CREATE TABLE(CTAS)
Hi Jark,
First of all, thank you for your very good advice!
The RTAS point you mentioned is
;> >Looking forward to further feedback from others.
>> >
>> >
>> >
>> >--
>> >
>> >Best regards,
>> >Mang Zhang
>> >
>> >
>> >
>> >
>> >
>> >At 2022-06-24 19:52:58, "ron&q
; >
> >
> >
> >
> >
> >At 2022-06-24 19:52:58, "ron" wrote:
> >>Thanks for godfrey further feedback, your suggestions are very good to
> me, the FLIP has updated according to your feedback. It will be very good
> if you look at it a
ing forward to further feedback from others.
>>
>>
>>> -----原始邮件-
>>> 发件人: "godfrey he"
>>> 发送时间: 2022-06-24 17:00:51 (星期五)
>>> 收件人: dev
>>> 抄送: "Yun Gao"
>>> 主题: Re: Re: Re: [DISCUSS] FLIP-218: Support SELECT
>> 发送时间: 2022-06-24 17:00:51 (星期五)
>> 收件人: dev
>> 抄送: "Yun Gao"
>> 主题: Re: Re: Re: [DISCUSS] FLIP-218: Support SELECT clause in CREATE
>> TABLE(CTAS)
>>
>> Hi all,
>>
>> Sorry for the late reply.
>>
>> >table.co
-06-24 17:00:51 (星期五)
> 收件人: dev
> 抄送: "Yun Gao"
> 主题: Re: Re: Re: [DISCUSS] FLIP-218: Support SELECT clause in CREATE
> TABLE(CTAS)
>
> Hi all,
>
> Sorry for the late reply.
>
> >table.cor-table-as-select.atomicity-enabled
> Regarding `cor`, thi
Hi all,
Sorry for the late reply.
>table.cor-table-as-select.atomicity-enabled
Regarding `cor`, this abbreviation is not commonly used.
>Create Table As Select(CTAS) feature depends on the serializability of the
>catalog. To quickly see if the catalog supports CTAS, we need to try to
>seriali
Hi Yun,
Thanks for your reply!
Through offline communication with Dalong, I updated the JobStatusHook part to
FLIP, looking forward to your feedback.
--
Best regards,
Mang Zhang
At 2022-05-31 14:34:25, "Yun Gao" wrote:
>Hi,
>
>Regarding the drop operation, with some offline discussion w
Hi Jing,
Thanks for your reply!
I recently updated the FLIP documentation, when you have time, you can look at
it again.
About Atomicity and Isolation, the main discussion point is on batch mode, to
be more specific is Hive/Spark.
My point of consideration is that according to the needs of the
Hi Mang,
Thanks for driving this! Finally there is a discussion about CTAS. It is
one of the most important features.
I agree with Jark that it would be good to be able to take care of both
atomicity and isolation, which lead to Spark DataSource v2. Would you like
to help me understand the connec
Hi,
Regarding the drop operation, with some offline discussion with Dalong and Zhu,
we think that listening in the client side might be problematic since it would
exit
after submitting the jobs in detached mode, thus the operation might need to
be in the JobMaster side.
For the listener interf
Hi Mang and Godfrey,
I'm okay with `Table#saveAs(tablePath)`, but it would be better to consider
overriding table options and create or replace table.
So we may also need to provide methods for the returned object:
- #option(key, value)
- #create()
- #createIfNotExist()
- #createOrReplace() //
Hi, Jark
Thanks for your reply and suggest!
>With this interface, we can easily support LEVEL-2 semantics by calling
>`Catalog#dropTable` in the
>`JobListener#onJobFailed`. We can also support LEVEL-3 by introducing
>`StagingTableCatalog` like Spark,
>calling `StagedTable#commitStagedChanges()` in
Hi, Godfrey
Thanks for your reply and suggest!
I adjusted the style of FLIP.
There are still two questions that need to be discussed:
1. The new API we introduced to CTAS, in Table or Tableenvalonment?
2. The Drop Table Operation in JM is everyone's consensus, more details design
at runtime need t
Hi Jark,
> "Table#createTableAs(tablePath)" seems a
>little strange to me.
`Table#createTableAs` is a bit misleading, I learn to Table#saveAs(tablePath).
Best,
Godfrey
Jark Wu 于2022年5月18日周三 23:09写道：
>
> Hi Godfrey,
>
> Regarding Table API for CTAS, "Table#createTableAs(tablePath)" seems a
> li
Hi Godfrey,
Regarding Table API for CTAS, "Table#createTableAs(tablePath)" seems a
little strange to me.
Usually, the parameter after AS should be the query, but the query is in
front of AS.
I slightly prefer a method on TableEnvironment besides "createTable" (i.e.
a special createTable with writi
Hi Mang,
Thanks for proposing this, CTAS is a very important API for batch users.
I think the key problem of this FLIP is the ACID semantics of the CTAS
operation.
We care most about two parts of the semantics:
1) Atomicity: the created table should be rolled back if the write is
failed.
2) Isola
Hi Mang,
Thanks for driving this FLIP.
Please follow the FLIP template[1] style, and the `Syntax ` is part of
the `Public API Changes` section.
‘Program research’ and 'Implementation Plan' are part of the `Proposed
Changes` section,
or move ‘Program research’ to the appendix.
> Providing methods
Hi everyone,
Can we identify if this proposed syntax is part of the SQL standard?
Best regards,
Martijn Visser
https://twitter.com/MartijnVisser82
https://github.com/MartijnVisser
On Fri, 29 Apr 2022 at 11:19, yuxia wrote:
> Thanks for for driving this work, it's to be a useful feature.
> Ab
Thanks for for driving this work, it's to be a useful feature.
About the flip-218, I have some questions.
1: Does our CTAS syntax support specify target table's schema including column
name and data type? I think it maybe a useful fature in case we want to change
the data types in target table i
21 matches
Mail list logo
