morrySnow commented on code in PR #1952: URL: https://github.com/apache/doris-website/pull/1952#discussion_r1942514201
########## docs/sql-manual/sql-statements/data-modification/load-and-export/BROKER-LOAD.md: ########## @@ -24,345 +23,286 @@ KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> + ## Description -This command is mainly used to import data on remote storage (such as S3, HDFS) through the Broker service process. +Broker Load is a data import method in Doris, primarily used to import large scale data from remote storage systems such as HDFS or S3. It is initiated through the MySQL API and is an asynchronous import method. The import progress and results can be queried using the `SHOW LOAD` statement. + +In earlier versions, S3 and HDFS Load relied on the Broker process. Now, data is read directly from the data source without relying on an additional Broker process. Nevertheless, due to the similar syntax, S3 Load, HDFS Load, and Broker Load are collectively referred to as Broker Load. + +## Syntax ```sql -LOAD LABEL load_label +LOAD LABEL [<db_name>.]<load_label> +( +[ { MERGE | APPEND | DELETE } ] +DATA INFILE ( -data_desc1[, data_desc2, ...] +"<file_path>"[, ...] +) +[ NEGATIVE ] +INTO TABLE `<table_name>` +[ PARTITION ( <partition_name> [ , ... ] ) ] +[ COLUMNS TERMINATED BY "<column_separator>" ] +[ LINES TERMINATED BY "<line_delimiter>" ] +[ FORMAT AS "<file_type>" ] +[ COMPRESS_TYPE AS "<compress_type>" ] +[ (<column_list>) ] +[ COLUMNS FROM PATH AS (<column_name> [ , ... ] ) ] +[ SET (<column_mapping>) ] +[ PRECEDING FILTER <predicate> ] +[ WHERE <predicate> ] +[ DELETE ON <expr> ] +[ ORDER BY <source_sequence> ] +[ PROPERTIES ("<key1>"="<value1>", ...) ] Review Comment: ```suggestion [ PROPERTIES ("<key>"="<value>" [, ...] ) ] ``` ########## docs/sql-manual/sql-statements/data-modification/load-and-export/MYSQL-LOAD.md: ########## @@ -24,62 +24,90 @@ specific language governing permissions and limitations under the License. --> - - ## Description -mysql-load: Import local data using the MySql client +Use the MySQL client to import local data files into Doris. MySQL Load is a synchronous import method, which returns the import result immediately after execution. You can determine whether the import is successful based on the return result of the `LOAD DATA` statement. MySQL Load can ensure the atomicity of a batch of import tasks, meaning that either all imports succeed or all fail. -``` +## Syntax + +```sql LOAD DATA -[LOCAL] -INFILE 'file_name' -INTO TABLE tbl_name -[PARTITION (partition_name [, partition_name] ...)] -[COLUMNS TERMINATED BY 'string'] -[LINES TERMINATED BY 'string'] -[IGNORE number {LINES | ROWS}] -[(col_name_or_user_var [, col_name_or_user_var] ...)] -[SET (col_name={expr | DEFAULT} [, col_name={expr | DEFAULT}] ...)] -[PROPERTIES (key1 = value1 [, key2=value2]) ] +[ LOCAL ] +INFILE "<file_name>" +INTO TABLE "<tbl_name>" +[ PARTITION (<partition_name> [, ... ]) ] +[ COLUMNS TERMINATED BY "<column_separator>" ] +[ LINES TERMINATED BY "<line_delimiter>" ] +[ IGNORE <number> {LINES | ROWS} ] +[ (<col_name_or_user_var> [, ... ] ) ] +[ SET (col_name={<expr> | DEFAULT} [, col_name={<expr> | DEFAULT}] ...) ] +[ PROPERTIES ("key1" = "value1" [ , ... ]) ] Review Comment: ```suggestion [ PROPERTIES ("<key>" = "<value>" [ , ... ]) ] ``` ########## docs/sql-manual/sql-statements/data-modification/load-and-export/SHOW-LOAD.md: ########## @@ -24,90 +24,109 @@ specific language governing permissions and limitations under the License. --> - - ## Description -This statement is used to display the execution of the specified import task +This statement is used to display the execution status of the specified import task. -grammar: +## Syntax ```sql SHOW LOAD -[FROM db_name] +[FROM <db_name>] [ WHERE - [LABEL [ = "your_label" | LIKE "label_matcher"]] - [STATE = ["PENDING"|"ETL"|"LOADING"|"FINISHED"|"CANCELLED"|]] + [LABEL = [ "<your_label>" | LIKE "<label_matcher>"]] + [ STATE = { " PENDING " | " ETL " | " LOADING " | " FINISHED " | " CANCELLED " } ] ] -[ORDER BY...] -[LIMIT limit][OFFSET offset]; +[ORDER BY [{ <col_name> | <expr> | <position> }]] Review Comment: ```suggestion [ORDER BY { <col_name> | <expr> | <position> }] ``` ########## docs/sql-manual/sql-statements/data-modification/load-and-export/SHOW-LOAD.md: ########## @@ -24,90 +24,109 @@ specific language governing permissions and limitations under the License. --> - - ## Description -This statement is used to display the execution of the specified import task +This statement is used to display the execution status of the specified import task. -grammar: +## Syntax ```sql SHOW LOAD -[FROM db_name] +[FROM <db_name>] [ WHERE - [LABEL [ = "your_label" | LIKE "label_matcher"]] - [STATE = ["PENDING"|"ETL"|"LOADING"|"FINISHED"|"CANCELLED"|]] + [LABEL = [ "<your_label>" | LIKE "<label_matcher>"]] + [ STATE = { " PENDING " | " ETL " | " LOADING " | " FINISHED " | " CANCELLED " } ] ] -[ORDER BY...] -[LIMIT limit][OFFSET offset]; +[ORDER BY [{ <col_name> | <expr> | <position> }]] +[LIMIT <limit>][OFFSET <offset>]; Review Comment: ```suggestion [LIMIT <limit>[OFFSET <offset>]]; ``` ########## docs/sql-manual/sql-statements/data-modification/load-and-export/MYSQL-LOAD.md: ########## @@ -24,62 +24,90 @@ specific language governing permissions and limitations under the License. --> - - ## Description -mysql-load: Import local data using the MySql client +Use the MySQL client to import local data files into Doris. MySQL Load is a synchronous import method, which returns the import result immediately after execution. You can determine whether the import is successful based on the return result of the `LOAD DATA` statement. MySQL Load can ensure the atomicity of a batch of import tasks, meaning that either all imports succeed or all fail. -``` +## Syntax + +```sql LOAD DATA -[LOCAL] -INFILE 'file_name' -INTO TABLE tbl_name -[PARTITION (partition_name [, partition_name] ...)] -[COLUMNS TERMINATED BY 'string'] -[LINES TERMINATED BY 'string'] -[IGNORE number {LINES | ROWS}] -[(col_name_or_user_var [, col_name_or_user_var] ...)] -[SET (col_name={expr | DEFAULT} [, col_name={expr | DEFAULT}] ...)] -[PROPERTIES (key1 = value1 [, key2=value2]) ] +[ LOCAL ] +INFILE "<file_name>" +INTO TABLE "<tbl_name>" +[ PARTITION (<partition_name> [, ... ]) ] +[ COLUMNS TERMINATED BY "<column_separator>" ] +[ LINES TERMINATED BY "<line_delimiter>" ] +[ IGNORE <number> {LINES | ROWS} ] +[ (<col_name_or_user_var> [, ... ] ) ] +[ SET (col_name={<expr> | DEFAULT} [, col_name={<expr> | DEFAULT}] ...) ] +[ PROPERTIES ("key1" = "value1" [ , ... ]) ] ``` -This statement is used to import data to the specified table. Unlike normal Load, this import method is a synchronous import. +## Required Parameters + +**1. `<file_name>`** + +> Specify the path of the local file, which can be either a relative or an absolute path. Currently, only a single file is supported, and multiple files are not supported. + +**2. `<tbl_name>`** + +> The table name can include the database name, as shown in the examples. If the database name is omitted, the current user's database will be used. + +## Optional Parameters + +**1. `LOCAL`** -This import method can still guarantee the atomicity of a batch of import tasks, either all data imports are successful or all fail. +> Specifying `LOCAL` indicates reading files from the client. Omitting it means reading files from the local storage of the FE server. The function of importing files from the FE server is disabled by default. You need to set `mysql_load_server_secure_path` on the FE node to specify a secure path to enable this function. -1. MySQL Load starts with the syntax `LOAD DATA`, without specifying `LABEL` -2. Specify `LOCAL` to read client side files. Not specified to read FE server side local files. Server side load was disabled by default. It can be enabled by setting a secure path in FE configuration `mysql_load_server_secure_path` -3. The local fill path will be filled after `INFILE`, which can be a relative path or an absolute path. Currently only a single file is supported, and multiple files are not supported -4. The table name after `INTO TABLE` can specify the database name, as shown in the case. It can also be omitted, and the database where the current user is located will be used. -5. `PARTITION` syntax supports specified partition to import -6. `COLUMNS TERMINATED BY` specifies the column separator -7. `LINES TERMINATED BY` specifies the line separator -8. `IGNORE num LINES` The user skips the header of the CSV and can skip any number of lines. This syntax can also be replaced by'IGNORE num ROWS ' -9. Column mapping syntax, please refer to the column mapping chapter of [Imported Data Transformation](../../../../data-operate/import/import-way/mysql-load-manual.md) -10. `PROPERTIES` parameter configuration, see below for details +**2. `<partition_name>`** -### PROPERTIES +> Multiple partitions can be specified for import, separated by commas. -1. max_filter_ratio: The maximum tolerable data ratio that can be filtered (for reasons such as data irregularity). Zero tolerance by default. Data irregularities do not include rows filtered out by where conditions. +**3. `<column_separator>`** -2. timeout: Specify the import timeout. in seconds. The default is 600 seconds. The setting range is from 1 second to 259200 seconds. +> Specify the column separator. -3. strict_mode: The user specifies whether to enable strict mode for this import. The default is off. +**4. `<line_delimiter>`** -4. timezone: Specify the time zone used for this import. The default is Dongba District. This parameter affects the results of all time zone-related functions involved in the import. +> Specify the line delimiter. -5. exec_mem_limit: Load memory limit. Default is 2GB. The unit is bytes. +**5. `IGNORE <number> { LINES | ROWS }`** -6. trim_double_quotes: Boolean type, The default value is false. True means that the outermost double quotes of each field in the load file are trimmed. +> Users can skip the header of the CSV file or any number of lines. This syntax can also be replaced with `IGNORE num ROWS`. -7. enclose: When the csv data field contains row delimiters or column delimiters, to prevent accidental truncation, single-byte characters can be specified as brackets for protection. For example, the column separator is ",", the bracket is "'", and the data is "a,'b,c'", then "b,c" will be parsed as a field. Note: when the bracket is `"`, trim\_double\_quotes must be set to true. +**6. `<col_name_or_user_var>`** -8. escape: Used to escape characters that appear in a csv field identical to the enclosing characters. For example, if the data is "a,'b,'c'", enclose is "'", and you want "b,'c to be parsed as a field, you need to specify a single-byte escape character, such as "\", and then modify the data to "a,' b,\'c'". +> Column mapping syntax. For specific parameters, refer to the column mapping section of [Data Transformation during Import](../../../../data-operate/import/import-way/mysql-load-manual.md). -## Example +**7. `<properties>`** Review Comment: 语法中已经没有 `<properties>` 了,需要和语法中对齐 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org