Hi, Mang +1 for completing the support for atomicity of CTAS, this is very useful in batch scenarios.
I have two questions: 1. naming wise: a) can we rename the `Catalog#getTwoPhaseCommitCreateTable` to `Catalog#twoPhaseCreateTable` (and we may add twoPhaseReplaceTable/twoPhaseCreateOrReplaceTable later) b) for the `TwoPhaseCommitCatalogTable`, may it be better using `TwoPhaseCatalogTable`? c) `TwoPhaseCommitCatalogTable#beginTransaction`, the word 'transaction' in the method name, which may remind users of the relevance of transaction support (however, it is not strictly so), so I suggest changing it to `begin` 2. Has this design been validated by any relevant Poc on hive or other catalogs? Best, Lincoln Lee liu ron <ron9....@gmail.com> 于2023年4月13日周四 10:17写道: > Hi, Mang > Atomicity is very important for CTAS, especially for batch jobs. This FLIP > is a continuation of FLIP-218, which is valuable for CTAS. > I just have one question, in the Motivation part of FLIP-218, we mentioned > three levels of atomicity semantics, can this current design do the same as > Spark's DataSource V2, which can guarantee both atomicity and isolation, > for example, can it be done by writing to Hive tables using CTAS? > > Best, > Ron > > Mang Zhang <zhangma...@163.com> 于2023年4月10日周一 11:03写道: > > > Hi, everyone > > > > > > > > > > I'd like to start a discussion about FLIP-305: Support atomic for CREATE > > TABLE AS SELECT(CTAS) statement [1]. > > > > > > > > > > CREATE TABLE AS SELECT(CTAS) statement has been support, but it's not > > atomic. It will create the table first before job running. If the job > > execution fails, or is cancelled, the table will not be dropped. > > > > > > > > > > So I want Flink to support atomic CTAS, where only the table is created > > when the Job succeeds. Improve user experience. > > > > > > > > > > Looking forward to your feedback. > > > > > > > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-305%3A+Support+atomic+for+CREATE+TABLE+AS+SELECT%28CTAS%29+statement > > > > > > > > > > > > > > > > > > > > > > -- > > > > Best regards, > > Mang Zhang >