Hi everyone.
If there are no other questions or concerns for the FLIP[1], I'd like to start
the vote next Monday (4.10).
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-302%3A+Support+TRUNCATE+TABLE+statement
Best regards,
Yuxia
----- 原始邮件 -----
发件人: "yuxia" <luoyu...@alumni.sjtu.edu.cn>
收件人: "dev" <dev@flink.apache.org>
发送时间: 星期五, 2023年 3 月 24日 上午 11:27:42
主题: Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement
Thanks all for your feedback.
@Shammon FY
My gut feeling is that the end user shouldn't care about whether it'll delete
direcotry or move to Trash directory with the TRUNCATE TABLE statement. They
only need to know it will delete all rows from a table.
To me, I think delete directory or move to trash is more likely to be a
behavior of external storage level instead of SQL statement level. In Hive, if
user configure Trash, it will then move files to trash for DROP statment.
Also, hardly did I see such usage with TRUNCATE TABLE statement in other
engines. What's more, to support it, we have to extend the TRUNCATE TABLE synax
which won't then compliant with SQL standard. I really don't want to do that
and I believe it'll make user confused if we do so.
@Hang
`TRUNCATE TABLE` is meant to delete all rows of a base table. So, it makes no
sense that table source implements it.
If user use TRUNCATE TABLE statement to truncate a table, the planner will only
try to
find the DynamicTableSink for the corresponding table.
@Ran Tao
1: Thanks for you reminder. I said it won't support view in the FLIP, but
forget to said temporary table is also not supported. Now, I add this part to
this FLIP.
2: Yes, I also considered to incldue it in this FLIP before. But as far as I
see, I haven't seen much usage of truncate table with partition. It's not as
useful as truncate table. So, I tend to keep this FLIP simple in here without
supporting truncate table with partition.
Also, seems for `truncate table with partition`, differnet engines may have
differernt syntax;
Hive[1]/Spark[2] use the following syntax:
TRUNCATE TABLE table_name [PARTITION partition_spec]
SqlServer[3] use the follwoing syntax:
TRUNCATE TABLE { database_name.schema_name.table_name | schema_name.table_name | table_name
} [ WITH ( PARTITIONS ( { <partition_number_expression> | <range> }
So, I'm tend to be cautious about it.
But I'm open to this. If there's any feedback or strong requirement, I don't
mind to add it in this FLIP.
If we do need it in some day, I can propose it in a new FLIP. It won't break
the current design.
As for concrete syntax in the FLIP, I think the current one is the concrete
syntax, we don't allow TABLE keyword to be optional.
3: Thanks for your reminder, I have updadted the FLIP for this.
[1]https://cwiki.apache.org/confluence/display/hive/languagemanual+ddl#LanguageManualDDL-TruncateTable
[2]https://spark.apache.org/docs/3.0.0-preview/sql-ref-syntax-ddl-truncate-table.html
[3]https://learn.microsoft.com/en-us/sql/t-sql/statements/truncate-table-transact-sql?view=sql-server-ver16
Best regards,
Yuxia
----- 原始邮件 -----
发件人: "Ran Tao" <chucheng...@gmail.com>
收件人: "dev" <dev@flink.apache.org>
发送时间: 星期四, 2023年 3 月 23日 下午 6:28:17
主题: Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement
Hi, yuxia.
Thanks for starting the discussion.
I think it's a nice improvement to support TRUNCATE TABLE statement because
many other mature engines supports it.
I have some questions.
1. because table has different types, whether we will support view or
temporary tables?
2. some other engines such as spark and hive support TRUNCATE TABLE with
partition. whether we will support？
btw, i think you need give the TRUNCATE TABLE concrete syntax in the FLIP
because some engines has different syntaxes.
for example, hive allow TRUNCATE TABLE be TRUNCATE [TABLE] which means
TABLE keyword can be optional.
3. The Proposed Changes try to use SqlToOperationConverter and run in
TableEnvironmentImpl#executeInternal.
I think it's out of date, the community is refactoring the conversion logic
from SqlNode to operation[1] and executions in TableEnvironmentImpl[2].
I suggest you can use new way to support it.
[1] https://issues.apache.org/jira/browse/FLINK-31464
[2] https://issues.apache.org/jira/browse/FLINK-31368
Best Regards,
Ran Tao
https://github.com/chucheng92
yuxia <luoyu...@alumni.sjtu.edu.cn> 于2023年3月22日周三 21:13写道：
Hi, devs.
I'd like to start a discussion about FLIP-302: Support TRUNCATE TABLE
statement [1].
The TRUNCATE TABLE statement is a SQL command that allows users to quickly
and efficiently delete all rows from a table without dropping the table
itself. This statement is commonly used in data warehouse, where large data
sets are frequently loaded and unloaded from tables.
So, this FLIP is meant to support TRUNCATE TABLE statement. M ore exactly,
this FLIP will bring Flink the TRUNCATE TABLE syntax and an interface with
which the coresponding connectors can implement their own logic for
truncating table.
Looking forwards to your feedback.
[1]: [
https://cwiki.apache.org/confluence/display/FLINK/FLIP-302%3A+Support+TRUNCATE+TABLE+statement
|
https://cwiki.apache.org/confluence/display/FLINK/FLIP-302%3A+Support+TRUNCATE+TABLE+statement
]
Best regards,
Yuxia
