Hi, all. I started a vote for this FLIP[1], please vote there[2] or ask additional questions here[3].
[1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-302%3A+Support+TRUNCATE+TABLE+statement+in+batch+mode [2] https://lists.apache.org/thread/fosvz0zcyfn6bp6vz2oxl45vq9qhkn2v [3] https://lists.apache.org/thread/m4r3wrd7p96wdst3nz3ncqzog6kf51cf Best regards, Yuxia ----- 原始邮件 ----- 发件人: "Jark Wu" <imj...@gmail.com> 收件人: "dev" <dev@flink.apache.org> 发送时间: 星期五, 2023年 4 月 14日 下午 11:04:58 主题: Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement Hi Yuxia, Thank you for the updating. That sounds good to me. Best, Jark > 2023年4月14日 19:00，yuxia <luoyu...@alumni.sjtu.edu.cn> 写道： > > Hi, Jark. > I'm expecting if the "executeTruncation" returns false, Flink will throw an > generic exception like "Fail to execute truncate table statement." > But the connector implementation can also throw more specific exception like > "Fail to execute truncate table statement for it table is been writing by > other jobs". > > But after think it over, I'm afraid of the connector implementation will > always return false to make Flink itself construnct the exception which maybe > not very useful for it provides > much less exception message instead of throwing more specific exception. > So I decide to change it to `void executeTruncation()` and reminder to throw > exception if truncate operation hasn't been executed successfully in the java > doc of the method. > I had updated this FLIP. > > > Best regards, > Yuxia > > ----- 原始邮件 ----- > 发件人: "Jark Wu" <imj...@gmail.com> > 收件人: "dev" <dev@flink.apache.org> > 发送时间: 星期五, 2023年 4 月 14日 下午 5:10:48 > 主题: Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement > > The FLIP looks good to me. +1 to start a vote. > > I just have a question: what will happen if the "executeTruncation" returns > false without any exceptions? > > Best, > Jark > > On Thu, 13 Apr 2023 at 19:59, Jing Ge <j...@ververica.com.invalid> wrote: > >> Thanks Yuxia for the clarification and FLIP update. The FLIP looks good! >> >> Best regards, >> Jing >> >> On Mon, Apr 10, 2023 at 3:51 AM yuxia <luoyu...@alumni.sjtu.edu.cn> wrote: >> >>> 1: >>> Actaully, considering the Flink's implementation, Flink just provides >>> Truncate Table syntax to help user simlify data management as said in >> this >>> FLIP and push the implementation of Truncate Table to external connector. >>> Normally, the effect of TRUENCATE TABLE is same as Drop Table + Create >>> Table. But the real difference/benefit depends on the implementation of >> the >>> external connector. >>> For example, for DROP Table statement, some external connectors may also >>> drop the view related or other things. >>> But for Truncate Table, the connectors may just delete all data without >>> other operations. >>> >>> >>> 2: >>> At very begining, I'm thinking about in which case user may want to >>> truncate a temporary table. >>> I thought users can always create a table in catalog(if the table doesn't >>> exist in a catalog) and truncate the table. So I tend not to expose it to >>> user. >>> But after I think it over again, I think it may be reasonable to support >>> truncate a temporary table for the case that user just want to delete all >>> datas from a table in an external storage without storing the metadata of >>> the table in a catalog so that the other user/session can't see the >> metada. >>> I think we can relax to the constraint to support truncate temporary >>> table. Now, I update it to the FLIP. >>> >>> >>> 3: >>> Thanks for your input, I agree that we can dicuss it in a different FLIP. >>> >>> >>> >>> Best regards, >>> Yuxia >>> >>> ----- 原始邮件 ----- >>> 发件人: "Jing Ge" <j...@ververica.com.INVALID> >>> 收件人: "dev" <dev@flink.apache.org> >>> 发送时间: 星期六, 2023年 4 月 08日 上午 3:05:11 >>> 主题: Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement >>> >>> Hi yuxia, >>> >>> Thanks for raising this topic. It is indeed a useful feature. +1 for >>> having it in Flink. I have some small questions and it would be great if >>> related information could be described in the FLIP. >>> >>> 1. Speaking of data warehouse use cases, what is the benefit of using >>> TRUNCATE table over DROP table + CREATE table IF NOT EXISTS with the >>> consideration of concrete Flink implementations? What would be the >>> suggestion for users to use TRUNCATE instead of DROP + CREATE... and >>> vise versa? >>> >>> 2. Since some engines support it, would you like to describe your >>> thought about why TRUNCATE table does not support temporary table? >>> >>> 3. The partition support is an important feature, afaic. It might >>> deserve a different FLIP and consider e.g.: TRUNCATE TABLE >>> tt_dw_usr_exp_xxx PARTITION(dt='20230303') and ALTER TABLE >>> tt_dw_usr_exp_xxx DROP IF EXISTS PARTITION(dt='20230303'). >>> >>> Looking forward to your thoughts. Thanks! >>> >>> Best regards, >>> >>> Jing >>> >>> On 4/7/23 05:04, Jingsong Li wrote: >>>> +1 for voting. >>>> >>>> Best, >>>> Jingsong >>>> >>>> On Thu, Apr 6, 2023 at 4:52 PM yuxia <luoyu...@alumni.sjtu.edu.cn> >>> wrote: >>>>> Hi everyone. >>>>> >>>>> If there are no other questions or concerns for the FLIP[1], I'd like >>> to start the vote next Monday (4.10). >>>>> >>>>> [1] >>> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-302%3A+Support+TRUNCATE+TABLE+statement >>>>> >>>>> Best regards, >>>>> Yuxia >>>>> >>>>> ----- 原始邮件 ----- >>>>> 发件人: "yuxia" <luoyu...@alumni.sjtu.edu.cn> >>>>> 收件人: "dev" <dev@flink.apache.org> >>>>> 发送时间: 星期五, 2023年 3 月 24日 上午 11:27:42 >>>>> 主题: Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement >>>>> >>>>> Thanks all for your feedback. >>>>> >>>>> @Shammon FY >>>>> My gut feeling is that the end user shouldn't care about whether it'll >>> delete direcotry or move to Trash directory with the TRUNCATE TABLE >>> statement. They only need to know it will delete all rows from a table. >>>>> To me, I think delete directory or move to trash is more likely to be >> a >>> behavior of external storage level instead of SQL statement level. In >> Hive, >>> if user configure Trash, it will then move files to trash for DROP >> statment. >>>>> Also, hardly did I see such usage with TRUNCATE TABLE statement in >>> other engines. What's more, to support it, we have to extend the TRUNCATE >>> TABLE synax which won't then compliant with SQL standard. I really don't >>> want to do that and I believe it'll make user confused if we do so. >>>>> >>>>> @Hang >>>>> `TRUNCATE TABLE` is meant to delete all rows of a base table. So, it >>> makes no sense that table source implements it. >>>>> If user use TRUNCATE TABLE statement to truncate a table, the planner >>> will only try to >>>>> find the DynamicTableSink for the corresponding table. >>>>> >>>>> @Ran Tao >>>>> 1: Thanks for you reminder. I said it won't support view in the FLIP, >>> but forget to said temporary table is also not supported. Now, I add this >>> part to this FLIP. >>>>> >>>>> 2: Yes, I also considered to incldue it in this FLIP before. But as >> far >>> as I see, I haven't seen much usage of truncate table with partition. >> It's >>> not as useful as truncate table. So, I tend to keep this FLIP simple in >>> here without supporting truncate table with partition. >>>>> Also, seems for `truncate table with partition`, differnet engines may >>> have differernt syntax; >>>>> Hive[1]/Spark[2] use the following syntax: >>>>> TRUNCATE TABLE table_name [PARTITION partition_spec] >>>>> >>>>> SqlServer[3] use the follwoing syntax: >>>>> TRUNCATE TABLE { database_name.schema_name.table_name | >>> schema_name.table_name | table_name } [ WITH ( PARTITIONS ( { >>> <partition_number_expression> | <range> } >>>>> So, I'm tend to be cautious about it. >>>>> >>>>> But I'm open to this. If there's any feedback or strong requirement, I >>> don't mind to add it in this FLIP. >>>>> If we do need it in some day, I can propose it in a new FLIP. It won't >>> break the current design. >>>>> >>>>> As for concrete syntax in the FLIP, I think the current one is the >>> concrete syntax, we don't allow TABLE keyword to be optional. >>>>> >>>>> 3: Thanks for your reminder, I have updadted the FLIP for this. >>>>> >>>>> >>>>> [1] >>> >> https://cwiki.apache.org/confluence/display/hive/languagemanual+ddl#LanguageManualDDL-TruncateTable >>>>> [2] >>> >> https://spark.apache.org/docs/3.0.0-preview/sql-ref-syntax-ddl-truncate-table.html >>>>> [3] >>> >> https://learn.microsoft.com/en-us/sql/t-sql/statements/truncate-table-transact-sql?view=sql-server-ver16 >>>>> >>>>> >>>>> >>>>> Best regards, >>>>> Yuxia >>>>> >>>>> ----- 原始邮件 ----- >>>>> 发件人: "Ran Tao" <chucheng...@gmail.com> >>>>> 收件人: "dev" <dev@flink.apache.org> >>>>> 发送时间: 星期四, 2023年 3 月 23日 下午 6:28:17 >>>>> 主题: Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement >>>>> >>>>> Hi, yuxia. >>>>> >>>>> Thanks for starting the discussion. >>>>> I think it's a nice improvement to support TRUNCATE TABLE statement >>> because >>>>> many other mature engines supports it. >>>>> >>>>> I have some questions. >>>>> 1. because table has different types, whether we will support view or >>>>> temporary tables? >>>>> >>>>> 2. some other engines such as spark and hive support TRUNCATE TABLE >> with >>>>> partition. whether we will support？ >>>>> btw, i think you need give the TRUNCATE TABLE concrete syntax in the >>> FLIP >>>>> because some engines has different syntaxes. >>>>> for example, hive allow TRUNCATE TABLE be TRUNCATE [TABLE] which means >>>>> TABLE keyword can be optional. >>>>> >>>>> 3. The Proposed Changes try to use SqlToOperationConverter and run in >>>>> TableEnvironmentImpl#executeInternal. >>>>> I think it's out of date, the community is refactoring the conversion >>> logic >>>>> from SqlNode to operation[1] and executions in >> TableEnvironmentImpl[2]. >>>>> I suggest you can use new way to support it. >>>>> >>>>> [1] https://issues.apache.org/jira/browse/FLINK-31464 >>>>> [2] https://issues.apache.org/jira/browse/FLINK-31368 >>>>> >>>>> Best Regards, >>>>> Ran Tao >>>>> https://github.com/chucheng92 >>>>> >>>>> >>>>> yuxia <luoyu...@alumni.sjtu.edu.cn> 于2023年3月22日周三 21:13写道： >>>>> >>>>>> Hi, devs. >>>>>> >>>>>> I'd like to start a discussion about FLIP-302: Support TRUNCATE TABLE >>>>>> statement [1]. >>>>>> >>>>>> The TRUNCATE TABLE statement is a SQL command that allows users to >>> quickly >>>>>> and efficiently delete all rows from a table without dropping the >> table >>>>>> itself. This statement is commonly used in data warehouse, where >> large >>> data >>>>>> sets are frequently loaded and unloaded from tables. >>>>>> So, this FLIP is meant to support TRUNCATE TABLE statement. M ore >>> exactly, >>>>>> this FLIP will bring Flink the TRUNCATE TABLE syntax and an interface >>> with >>>>>> which the coresponding connectors can implement their own logic for >>>>>> truncating table. >>>>>> >>>>>> Looking forwards to your feedback. >>>>>> >>>>>> [1]: [ >>>>>> >>> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-302%3A+Support+TRUNCATE+TABLE+statement >>>>>> | >>>>>> >>> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-302%3A+Support+TRUNCATE+TABLE+statement >>>>>> ] >>>>>> >>>>>> >>>>>> Best regards, >>>>>> Yuxia >>>>>> >>> >>
