This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/dev/pr-12347-0064881f158af8dff33b1c9c7fbc1f4660361b66 in repository https://gitbox.apache.org/repos/asf/seatunnel.git
commit 94e8490971b969e9c60a280cd686edebd34d7209 Author: Jast <[email protected]> AuthorDate: Sun Oct 11 09:45:57 2026 +0000 [Docs] Fix connector option defaults, malformed tables, broken links, and add missing zh Splunk doc (#12347) Co-authored-by: David Zollo <[email protected]> Co-authored-by: Claude Sonnet 5.5 <[email protected]> --- docs/en/connectors/source/BosFile.md | 2 +- docs/en/connectors/source/CosFile.md | 2 +- docs/en/connectors/source/FtpFile.md | 2 +- docs/en/connectors/source/GcsFile.md | 2 +- docs/en/connectors/source/HdfsFile.md | 4 +-- docs/en/connectors/source/LocalFile.md | 4 +-- docs/en/connectors/source/OneSignal.md | 2 +- docs/en/connectors/source/OssJindoFile.md | 2 +- docs/en/connectors/source/S3File.md | 2 +- docs/en/connectors/source/SftpFile.md | 2 +- docs/en/connectors/source/Socket.md | 4 +-- docs/en/developer/contribute-plugin.md | 2 +- docs/en/developer/contribute-transform-v2-guide.md | 4 +-- docs/en/developer/how-to-create-your-connector.md | 2 +- docs/en/faq.md | 2 +- .../connectors/changelog/connector-http-splunk.md | 2 +- docs/zh/connectors/source/BosFile.md | 2 +- docs/zh/connectors/source/GcsFile.md | 2 +- docs/zh/connectors/source/Hive.md | 38 +++++++++++----------- docs/zh/connectors/source/LocalFile.md | 6 +++- docs/zh/connectors/source/OneSignal.md | 2 +- docs/zh/connectors/source/OssJindoFile.md | 2 +- docs/zh/developer/contribute-plugin.md | 2 +- docs/zh/developer/contribute-transform-v2-guide.md | 4 +-- docs/zh/developer/how-to-create-your-connector.md | 2 +- docs/zh/faq.md | 2 +- 26 files changed, 53 insertions(+), 49 deletions(-) diff --git a/docs/en/connectors/source/BosFile.md b/docs/en/connectors/source/BosFile.md index 563cf3fc39..e4492b81b7 100644 --- a/docs/en/connectors/source/BosFile.md +++ b/docs/en/connectors/source/BosFile.md @@ -63,7 +63,7 @@ To use this connector you need to put `bos-hdfs-sdk` (>= 1.0.4-community) into ` | endpoint | string | yes | - | | read_columns | list | no | - | | read_partitions | list | no | - | -| delimiter/field_delimiter | string | no | \001 for text and , for csv | +| delimiter/field_delimiter | string | no | \001 | | row_delimiter | string | no | \n | | parse_partition_from_path | boolean | no | true | | skip_header_row_number | long | no | 0 | diff --git a/docs/en/connectors/source/CosFile.md b/docs/en/connectors/source/CosFile.md index f312fa8ceb..1e7a1ff232 100644 --- a/docs/en/connectors/source/CosFile.md +++ b/docs/en/connectors/source/CosFile.md @@ -63,7 +63,7 @@ To use this connector you need put hadoop-cos-{hadoop.version}-{version}.jar and | region | string | yes | - | | read_columns | list | no | - | | read_partitions | list | no | - | -| delimiter/field_delimiter | string | no | \001 for text and , for csv | +| delimiter/field_delimiter | string | no | \001 | | row_delimiter | string | no | \n | | parse_partition_from_path | boolean | no | true | | skip_header_row_number | long | no | 0 | diff --git a/docs/en/connectors/source/FtpFile.md b/docs/en/connectors/source/FtpFile.md index 13d5ca33f6..ab53f543de 100644 --- a/docs/en/connectors/source/FtpFile.md +++ b/docs/en/connectors/source/FtpFile.md @@ -59,7 +59,7 @@ If you use SeaTunnel Engine, It automatically integrated the hadoop jar when you | connection_mode | string | no | active_local | | remote_verification_enabled | boolean | no | true | | control_encoding | string | no | UTF-8 | -| delimiter/field_delimiter | string | no | \001 for text and , for csv | +| delimiter/field_delimiter | string | no | \001 | | row_delimiter | string | no | \n | | read_columns | list | no | - | | read_partitions | list | no | - | diff --git a/docs/en/connectors/source/GcsFile.md b/docs/en/connectors/source/GcsFile.md index 7fdcc4e1f9..260b3147ed 100644 --- a/docs/en/connectors/source/GcsFile.md +++ b/docs/en/connectors/source/GcsFile.md @@ -61,7 +61,7 @@ The explicit `service_account_key_file` option takes precedence over the corresp | schema | config | conditional | - | Required for `text`, `json`, `excel`, `csv`, and `xml`. See [Schema Feature](../../introduction/concepts/schema-feature.md). | | read_columns | list | no | - | Columns to project from the source. | | read_partitions | list | no | - | The partitions that the user wants to read, e.g. `["year=2024"]`. When set, only these partitions are read. | -| field_delimiter | string | no | `\001` for text, `,` for CSV | Field delimiter for text and CSV files. `delimiter` is an alias. | +| field_delimiter | string | no | `\001` | Field delimiter for text and CSV files. `delimiter` is an alias. | | row_delimiter | string | no | `\n` | Row delimiter for text files. | | skip_header_row_number | long | no | `0` | Number of initial text or CSV rows to skip. | | encoding | string | no | `UTF-8` | Character encoding for text, JSON, CSV, and XML files. | diff --git a/docs/en/connectors/source/HdfsFile.md b/docs/en/connectors/source/HdfsFile.md index 9cb0692f13..df272f0aa8 100644 --- a/docs/en/connectors/source/HdfsFile.md +++ b/docs/en/connectors/source/HdfsFile.md @@ -53,13 +53,13 @@ Read data from hdfs file system. | Name | Type | Required | Default | Description | |----------------------------|---------|----------|-----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | path | string | yes | - | The source file path. | -| tables_configs | list | no | - | Configure multiple HDFS source tables in one source block. Each item has the same options as a single-table `HdfsFile` source, and can set its downstream table name through `schema.table`. | +| tables_configs | list | no | - | Configure multiple HDFS source tables in one source block. Each item has the same options as a single-table `HdfsFile` source, and can set its downstream table name through `schema.table`. | | file_format_type | string | yes | - | We supported as the following file types:`text` `csv` `parquet` `orc` `json` `excel` `xml` `binary` `markdown` `pdf`.Please note that, The final file name will end with the file_format's suffix, the suffix of the text file is `txt`. | | fs.defaultFS | string | yes | - | The hadoop cluster address that start with `hdfs://`, for example: `hdfs://hadoopcluster` | | read_columns | list | no | - | The read column list of the data source, user can use it to implement field projection.The file type supported column projection as the following shown:[text,json,csv,orc,parquet,excel,xml].Tips: If the user wants to use this feature when reading `text` `json` `csv` files, the schema option must be configured. | | read_partitions | list | no | - | The partitions that the user wants to read, e.g. `["year=2024"]`. When set, only these partitions are read. | | hdfs_site_path | string | no | - | The path of `hdfs-site.xml`, used to load ha configuration of namenodes | -| delimiter/field_delimiter | string | no | \001 for text and , for csv | Field delimiter, used to tell connector how to slice and dice fields when reading text files. default `\001`, the same as hive's default delimiter | +| delimiter/field_delimiter | string | no | \001 | Field delimiter, used to tell connector how to slice and dice fields when reading text files. default `\001`, the same as hive's default delimiter | | row_delimiter | string | no | \n | Row delimiter, used to tell connector how to slice and dice rows when reading text files. default `\n` | | parse_partition_from_path | boolean | no | true | Control whether parse the partition keys and values from file path. For example if you read a file from path `hdfs://hadoop-cluster/tmp/seatunnel/parquet/name=tyrantlucifer/age=26`. Every record data from file will be added these two fields:[name:tyrantlucifer,age:26].Tips:Do not define partition fields in schema option. | | date_format | string | no | yyyy-MM-dd | Date type format, used to tell connector how to convert string to date, supported as the following formats:`yyyy-MM-dd` `yyyy.MM.dd` `yyyy/MM/dd` default `yyyy-MM-dd`.Date type format, used to tell connector how to convert string to date, supported as the following formats:`yyyy-MM-dd` `yyyy.MM.dd` `yyyy/MM/dd` default `yyyy-MM-dd` | diff --git a/docs/en/connectors/source/LocalFile.md b/docs/en/connectors/source/LocalFile.md index 88a72c7ef7..413101e451 100644 --- a/docs/en/connectors/source/LocalFile.md +++ b/docs/en/connectors/source/LocalFile.md @@ -57,7 +57,7 @@ If you use SeaTunnel Engine, It automatically integrated the hadoop jar when you | file_format_type | string | yes | - | | read_columns | list | no | - | | read_partitions | list | no | - | -| delimiter/field_delimiter | string | no | \001 for text and , for csv | +| delimiter/field_delimiter | string | no | \001 | | row_delimiter | string | no | \n | | parse_partition_from_path | boolean | no | true | | date_format | string | no | yyyy-MM-dd | @@ -94,7 +94,7 @@ If you use SeaTunnel Engine, It automatically integrated the hadoop jar when you | retention_max_age | string | no | - | | retention_check_interval | string | no | 1H | | common-options | | no | - | -| tables_configs | list | no | used to define a multiple table task | +| tables_configs | list | no | - | | file_filter_modified_start | string | no | - | | file_filter_modified_end | string | no | - | | enable_file_split | boolean | no | false | diff --git a/docs/en/connectors/source/OneSignal.md b/docs/en/connectors/source/OneSignal.md index 31cbee801c..c72e5ea591 100644 --- a/docs/en/connectors/source/OneSignal.md +++ b/docs/en/connectors/source/OneSignal.md @@ -29,7 +29,7 @@ Use this connector to ingest OneSignal resources such as apps, players, segments | headers | Map | No | - | Extra HTTP headers. Do not put `Authorization` here unless you want to override the header generated from `password`. | | params | Map | No | - | HTTP query parameters, such as `limit`, `offset`, or other OneSignal API parameters. | | body | String | No | - | HTTP request body. Useful for endpoints that accept a JSON payload. | -| format | String | No | json | Response format. Use `json` with `schema` to read OneSignal JSON as SeaTunnel rows with named fields. Use `text` to keep the raw response. | +| format | String | No | text | Response format. Use `json` with `schema` to read OneSignal JSON as SeaTunnel rows with named fields. Use `text` to keep the raw response. | | schema | Config | No | - | Output row structure. Required when `format = "json"`. See [Schema Feature](../../introduction/concepts/schema-feature.md). | | schema.fields | Config | No | - | Field names and SeaTunnel data types used to parse the JSON response. | | json_field | Config | No | - | Field-level JSONPath mapping. Use it with `schema` when each output field lives at a different JSON path. | diff --git a/docs/en/connectors/source/OssJindoFile.md b/docs/en/connectors/source/OssJindoFile.md index 202f85d598..ffd62381c1 100644 --- a/docs/en/connectors/source/OssJindoFile.md +++ b/docs/en/connectors/source/OssJindoFile.md @@ -67,7 +67,7 @@ It only supports hadoop version **2.9.X+**. | endpoint | string | yes | - | | read_columns | list | no | - | | read_partitions | list | no | - | -| delimiter/field_delimiter | string | no | \001 for text and , for csv | +| delimiter/field_delimiter | string | no | \001 | | row_delimiter | string | no | \n | | parse_partition_from_path | boolean | no | true | | date_format | string | no | yyyy-MM-dd | diff --git a/docs/en/connectors/source/S3File.md b/docs/en/connectors/source/S3File.md index 4d103dfbed..9a253ac423 100644 --- a/docs/en/connectors/source/S3File.md +++ b/docs/en/connectors/source/S3File.md @@ -242,7 +242,7 @@ If you assign file type to `parquet` `orc`, schema option not required, connecto | access_key | string | no | - | Only used when `fs.s3a.aws.credentials.provider = org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider ` [...] | secret_key | string | no | - | Only used when `fs.s3a.aws.credentials.provider = org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider ` [...] | hadoop_s3_properties | map | no | - | If you need to add other option, you could add it here and refer to this [link](https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html) [...] -| delimiter/field_delimiter | string | no | \001 for text and , for csv | Field delimiter, used to tell connector how to slice and dice fields when reading text files. Default `\001`, the same as hive's default delimiter. [...] +| delimiter/field_delimiter | string | no | \001 | Field delimiter, used to tell connector how to slice and dice fields when reading text files. Default `\001`, the same as hive's default delimiter. [...] | row_delimiter | string | no | \n | Row delimiter, used to tell connector how to slice and dice rows when reading text files. Default `\n`. [...] | parse_partition_from_path | boolean | no | true | Control whether parse the partition keys and values from file path. For example if you read a file from path `s3n://hadoop-cluster/tmp/seatunnel/parquet/name=tyrantlucifer/age=26`. Every record data from file will be added these two fields: name="tyrantlucifer", age=16 [...] | date_format | string | no | yyyy-MM-dd | Date type format, used to tell connector how to convert string to date, supported as the following formats:`yyyy-MM-dd` `yyyy.MM.dd` `yyyy/MM/dd`. default `yyyy-MM-dd` [...] diff --git a/docs/en/connectors/source/SftpFile.md b/docs/en/connectors/source/SftpFile.md index b14917456d..fc6f426032 100644 --- a/docs/en/connectors/source/SftpFile.md +++ b/docs/en/connectors/source/SftpFile.md @@ -94,7 +94,7 @@ The File does not have a specific type list, and we can indicate which SeaTunnel | file_format_type | String | Yes | - | Please check #file_format_type below. Supported file types: `text` `csv` `parquet` `orc` `json` `excel` `xml` `binary` `markdown` `pdf` [...] | file_filter_pattern | String | No | - | Filter pattern, which used for filtering files. | | filename_extension | string | no | - | Filter filename extension, which used for filtering files with specific extension. Example: `csv` `.txt` `json` `.xml`. | -| delimiter/field_delimiter | String | No | \001 for text and ',' for csv | **delimiter** parameter will deprecate after version 2.3.5, please use **field_delimiter** instead. <br/> Field delimiter, used to tell connector how to slice and dice fields when reading text files. <br/> Default `\001`, the same as hive's default delimiter | +| delimiter/field_delimiter | String | No | \001 | **delimiter** parameter will deprecate after version 2.3.5, please use **field_delimiter** instead. <br/> Field delimiter, used to tell connector how to slice and dice fields when reading text files. <br/> Default `\001`, the same as hive's default delimiter | | row_delimiter | string | no | \n | Row delimiter, used to tell connector how to slice and dice rows when reading text files. <br/> Default `\n` | | parse_partition_from_path | Boolean | No | true | Control whether parse the partition keys and values from file path <br/> For example if you read a file from path `oss://hadoop-cluster/tmp/seatunnel/parquet/name=tyrantlucifer/age=26` <br/> Every record data from file will be added these two fields: <br/> name age <br/> tyrantlucifer 26 <br/> Tips: **Do not define partition fields in schema option** | | date_format | String | No | yyyy-MM-dd | Date type format, used to tell connector how to convert string to date, supported as the following formats: <br/> `yyyy-MM-dd` `yyyy.MM.dd` `yyyy/MM/dd` <br/> default `yyyy-MM-dd` | diff --git a/docs/en/connectors/source/Socket.md b/docs/en/connectors/source/Socket.md index 5c15aa4978..ce07d45a83 100644 --- a/docs/en/connectors/source/Socket.md +++ b/docs/en/connectors/source/Socket.md @@ -44,8 +44,8 @@ Socket source reads each incoming line as a string record. | Name | Type | Required | Default | Description | |----------------|---------|----------|---------|--------------------------------------------------------------------------------------------------------------------| -| host | String | Yes | _ | socket server host | -| port | Integer | Yes | _ | socket server port | +| host | String | Yes | - | socket server host | +| port | Integer | Yes | - | socket server port | | common-options | | no | - | Source plugin common parameters, please refer to [Source Common Options](../common-options/source-common-options.md) for details. | :::tip diff --git a/docs/en/developer/contribute-plugin.md b/docs/en/developer/contribute-plugin.md index fd556fe376..91aee353fa 100644 --- a/docs/en/developer/contribute-plugin.md +++ b/docs/en/developer/contribute-plugin.md @@ -22,4 +22,4 @@ source of truth. - [Source Connector Development](./source-connector-development.md) - [Sink Connector Development](./sink-connector-development.md) - [Set Up Develop Environment](./setup.md) -- External repository guide: [Connector-v2 Contribution Guide](../../../seatunnel-connectors-v2/README.md) +- External repository guide: [Connector-v2 Contribution Guide](https://github.com/apache/seatunnel/blob/dev/seatunnel-connectors-v2/README.md) diff --git a/docs/en/developer/contribute-transform-v2-guide.md b/docs/en/developer/contribute-transform-v2-guide.md index f5cd3c8d8d..2fd4d767e2 100644 --- a/docs/en/developer/contribute-transform-v2-guide.md +++ b/docs/en/developer/contribute-transform-v2-guide.md @@ -50,10 +50,10 @@ Transforms are easier to review when the code, docs, and examples change togethe 2. [Transforms Catalog](../transforms) 3. [Transform Common Options](../transforms/common-options/common-options.md) 4. [Core API Design](../architecture/core-api-design.md) -5. [seatunnel-transforms-v2 README](../../../seatunnel-transforms-v2/README.md) +5. [seatunnel-transforms-v2 README](https://github.com/apache/seatunnel/blob/dev/seatunnel-transforms-v2/README.md) ## When You Need The Repository-Level Guide For module-specific details such as transform module conventions or examples inside the `seatunnel-transforms-v2` tree, continue with the repository guide: -- [Transform-V2 Contribution Guide](../../../seatunnel-transforms-v2/README.md) +- [Transform-V2 Contribution Guide](https://github.com/apache/seatunnel/blob/dev/seatunnel-transforms-v2/README.md) diff --git a/docs/en/developer/how-to-create-your-connector.md b/docs/en/developer/how-to-create-your-connector.md index 19693ed931..38d59f5cae 100644 --- a/docs/en/developer/how-to-create-your-connector.md +++ b/docs/en/developer/how-to-create-your-connector.md @@ -1,6 +1,6 @@ # Develop Your Own Connector -If you want to develop your own connector for the new SeaTunnel connector API (Connector V2), please check [here](../../../seatunnel-connectors-v2/README.md). +If you want to develop your own connector for the new SeaTunnel connector API (Connector V2), please check [here](https://github.com/apache/seatunnel/blob/dev/seatunnel-connectors-v2/README.md). ## Start Here diff --git a/docs/en/faq.md b/docs/en/faq.md index 0d19da241e..9743ad3c25 100644 --- a/docs/en/faq.md +++ b/docs/en/faq.md @@ -128,4 +128,4 @@ For more details, see: [lightbend/config#456](https://github.com/lightbend/confi SeaTunnel features a highly abstracted and well-structured architecture, making it an excellent choice for learning big data architecture. You can start by exploring and debugging the `seatunnel-examples` module: `SeaTunnelEngineLocalExample.java`. For more details, refer to the [SeaTunnel Contribution Guide](https://seatunnel.apache.org/docs/developer/setup). ## Do I need to understand all of SeaTunnel’s source code if I want to develop my own source, sink, or transform? -No, you only need to focus on the interfaces for source, sink, and transform. If you want to develop your own connector (Connector V2) for the SeaTunnel API, refer to the **[Connector Development Guide](../../seatunnel-connectors-v2/README.md)**. +No, you only need to focus on the interfaces for source, sink, and transform. If you want to develop your own connector (Connector V2) for the SeaTunnel API, refer to the **[Connector Development Guide](https://github.com/apache/seatunnel/blob/dev/seatunnel-connectors-v2/README.md)**. diff --git a/docs/zh/connectors/changelog/connector-http-splunk.md b/docs/zh/connectors/changelog/connector-http-splunk.md index c2e6a88eea..2433b47827 100644 --- a/docs/zh/connectors/changelog/connector-http-splunk.md +++ b/docs/zh/connectors/changelog/connector-http-splunk.md @@ -1,3 +1,3 @@ | Change | Commit | Version | | --- | --- | --- | -| [Feature][connector-http-splunk] Add Splunk source connector | https://github.com/apache/seatunnel | dev | \ No newline at end of file +| [Feature][connector-http-splunk] Add Splunk source connector | https://github.com/apache/seatunnel | dev | diff --git a/docs/zh/connectors/source/BosFile.md b/docs/zh/connectors/source/BosFile.md index f6d76d290d..bb4852b1d2 100644 --- a/docs/zh/connectors/source/BosFile.md +++ b/docs/zh/connectors/source/BosFile.md @@ -63,7 +63,7 @@ import ChangeLog from '../changelog/connector-file-bos.md'; | endpoint | string | 是 | - | | read_columns | list | 否 | - | | read_partitions | list | 否 | - | -| delimiter/field_delimiter | string | 否 | text 为 \001，csv 为 , | +| delimiter/field_delimiter | string | 否 | \001 | | row_delimiter | string | 否 | \n | | parse_partition_from_path | boolean | 否 | true | | skip_header_row_number | long | 否 | 0 | diff --git a/docs/zh/connectors/source/GcsFile.md b/docs/zh/connectors/source/GcsFile.md index 170c10daab..0aa7f4a74e 100644 --- a/docs/zh/connectors/source/GcsFile.md +++ b/docs/zh/connectors/source/GcsFile.md @@ -51,7 +51,7 @@ import ChangeLog from '../changelog/connector-file-gcs.md'; | schema | config | 条件必填 | - | `text`、`json`、`excel`、`csv` 和 `xml` 格式需要配置。参见 [Schema 功能](../../introduction/concepts/schema-feature.md)。 | | read_columns | list | 否 | - | 从数据源投影的列。 | | read_partitions | list | 否 | - | 用户希望读取的分区列表，例如 `["year=2024"]`。设置后仅读取这些分区。 | -| field_delimiter | string | 否 | text 为 `\001`，CSV 为 `,` | text 和 CSV 的字段分隔符，`delimiter` 是其别名。 | +| field_delimiter | string | 否 | `\001` | text 和 CSV 的字段分隔符，`delimiter` 是其别名。 | | row_delimiter | string | 否 | `\n` | text 文件的行分隔符。 | | skip_header_row_number | long | 否 | `0` | 跳过 text 或 CSV 文件开头的行数。 | | encoding | string | 否 | `UTF-8` | text、JSON、CSV 和 XML 的字符编码。 | diff --git a/docs/zh/connectors/source/Hive.md b/docs/zh/connectors/source/Hive.md index a7a57a64ac..a5bd5880c6 100644 --- a/docs/zh/connectors/source/Hive.md +++ b/docs/zh/connectors/source/Hive.md @@ -48,25 +48,25 @@ import ChangeLog from '../changelog/connector-hive.md'; ## 选项 -| 名称 | 类型 | 必需 | 默认值 | -|-----------------------|--------|------|---------| -| table_name | string | 否 | 单表模式必填 | -| table_list | array | 否 | 已废弃，请使用 `tables_configs` | -| tables_configs | array | 否 | 多表读取时使用的 Hive 表配置列表，每项可覆盖根配置中的任意选项。 | -| use_regex | boolean| 否 | false | -| metastore_uri | string | 否 | 单表模式必填 | -| krb5_path | string | 否 | /etc/krb5.conf | -| kerberos_principal | string | 否 | - | -| kerberos_keytab_path | string | 否 | - | -| hdfs_site_path | string | 否 | - | -| hive_site_path | string | 否 | - | -| hive.hadoop.conf | Map | 否 | - | -| hive.hadoop.conf-path | string | 否 | - | -| remote_user | string | 否 | - | -| read_partitions | list | 否 | - | -| read_columns | list | 否 | - | -| compress_codec | string | 否 | none | -| common-options | | 否 | - | +| 名称 | 类型 | 必需 | 默认值 | 描述 | +|-----------------------|--------|------|---------|------| +| table_name | string | 否 | 单表模式必填 | 目标 Hive 表名，格式为 `db1.table1`。当 `use_regex = true` 时，该字段使用 `数据库正则.表正则` 匹配多张表。 | +| table_list | array | 否 | - | 已废弃的多表配置列表。新作业请使用 `tables_configs`。仅为向后兼容保留，未来版本将移除。 | +| tables_configs | array | 否 | - | 多表读取时使用的 Hive 表配置列表，每项可覆盖根配置中的任意选项。 | +| use_regex | boolean| 否 | false | 将 `table_name` 视为正则表达式以匹配多张表。在根级别以及 `table_list` / `tables_configs` 的每一项中均可使用。 | +| metastore_uri | string | 否 | 单表模式必填 | Hive metastore URI。多个以逗号分隔的值可启用 HA 故障转移，空白字符会被忽略。 | +| krb5_path | string | 否 | /etc/krb5.conf | Kerberos 认证使用的 `krb5.conf` 文件路径。 | +| kerberos_principal | string | 否 | - | 访问 Hive Metastore / HDFS 的 Kerberos principal。 | +| kerberos_keytab_path | string | 否 | - | 与 `kerberos_principal` 配套的 keytab 文件路径。 | +| hdfs_site_path | string | 否 | - | `hdfs-site.xml` 的本地路径，用于加载 HDFS HA 配置。新作业不建议使用，请改用 `hive.hadoop.conf` 或 `hive.hadoop.conf-path`。 | +| hive_site_path | string | 否 | - | `hive-site.xml` 的本地路径。 | +| hive.hadoop.conf | Map | 否 | - | 内联的 Hadoop 配置项（等价于 `core-site.xml` / `hdfs-site.xml` / `hive-site.xml` 中的条目）。 | +| hive.hadoop.conf-path | string | 否 | - | 包含 `core-site.xml`、`hdfs-site.xml` 和 `hive-site.xml` 的目录。 | +| remote_user | string | 否 | - | 未启用 Kerberos 时连接 HDFS / Hive 存储使用的 Hadoop 远程用户名。 | +| read_partitions | list | 否 | - | 限制读取的分区子集。所有条目的目录层级深度必须一致。 | +| read_columns | list | 否 | - | 列投影列表。仅读取列出的字段。 | +| compress_codec | string | 否 | none | text / CSV / JSON 输出的压缩编解码器，支持 `lzo` 和 `none`。Parquet / ORC 会自动识别压缩。 | +| common-options | | 否 | - | Source 插件通用参数，详情请参见 [Source Common Options](../common-options/source-common-options.md)。 | ### table_name [string] diff --git a/docs/zh/connectors/source/LocalFile.md b/docs/zh/connectors/source/LocalFile.md index b7c4f7cf49..396274b888 100644 --- a/docs/zh/connectors/source/LocalFile.md +++ b/docs/zh/connectors/source/LocalFile.md @@ -94,7 +94,7 @@ import ChangeLog from '../changelog/connector-file-local.md'; | retention_max_age | string | 否 | - | | retention_check_interval | string | 否 | 1H | | common-options | | 否 | - | -| tables_configs | list | 否 | 用于定义多表任务 | +| tables_configs | list | 否 | - | | file_filter_modified_start | string | 否 | - | | file_filter_modified_end | string | 否 | - | | enable_file_split | boolean | 否 | false | @@ -641,6 +641,10 @@ compare_mode = "len_mtime" - 读取具有不断演化的 schema 的文件，且希望 schema 推断使用最新的文件 - 需要按时间顺序处理文件 +### metalake_type [string] + +Metalake 服务类型，目前支持 `gravitino`。 + ### 通用选项 数据源插件通用参数，请参阅 [数据源通用选项](../common-options/source-common-options.md) 了解详情 diff --git a/docs/zh/connectors/source/OneSignal.md b/docs/zh/connectors/source/OneSignal.md index fdf154b03d..217e736fd5 100644 --- a/docs/zh/connectors/source/OneSignal.md +++ b/docs/zh/connectors/source/OneSignal.md @@ -29,7 +29,7 @@ OneSignal 源连接器用于从 OneSignal 的 REST API 读取数据。它基于 | headers | Map | 否 | - | 额外的 HTTP 请求头。除非需要覆盖由 `password` 生成的头，否则不要在这里配置 `Authorization`。 | | params | Map | 否 | - | HTTP 查询参数，例如 `limit`、`offset` 等 OneSignal API 参数。 | | body | String | 否 | - | HTTP 请求体，对支持 JSON 负载的接口有用。 | -| format | String | 否 | json | 响应格式，`json` 时需要配合 `schema`；`text` 时返回原始响应。 | +| format | String | 否 | text | 响应格式，`json` 时需要配合 `schema`；`text` 时返回原始响应。 | | schema | Config | 否 | - | 输出数据结构，`format = "json"` 时必填。详见 [Schema 特性](../../introduction/concepts/schema-feature.md)。 | | schema.fields | Config | 否 | - | 字段名与 SeaTunnel 数据类型，用于解析 JSON 响应。 | | json_field | Config | 否 | - | 字段级 JSONPath 映射，与 `schema` 配合使用。 | diff --git a/docs/zh/connectors/source/OssJindoFile.md b/docs/zh/connectors/source/OssJindoFile.md index c94a3baafb..b94af55993 100644 --- a/docs/zh/connectors/source/OssJindoFile.md +++ b/docs/zh/connectors/source/OssJindoFile.md @@ -67,7 +67,7 @@ import ChangeLog from '../changelog/connector-file-oss-jindo.md'; | endpoint | string | 是 | - | OSS 文件系统的端点 | | read_columns | list | 否 | - | 数据源的读取列列表 | | read_partitions | list | 否 | - | 用户希望读取的分区列表，例如 `["year=2024"]`。设置后仅读取这些分区。 | -| delimiter/field_delimiter | string | 否 | \001 for text and , for csv | 字段分隔符 | +| delimiter/field_delimiter | string | 否 | \001 | 字段分隔符 | | row_delimiter | string | 否 | \n | 行分隔符 | | parse_partition_from_path | boolean | 否 | true | 控制是否从文件路径解析分区键和值 | | date_format | string | 否 | yyyy-MM-dd | 日期类型格式 | diff --git a/docs/zh/developer/contribute-plugin.md b/docs/zh/developer/contribute-plugin.md index 983b57e2c6..e8fd4ac774 100644 --- a/docs/zh/developer/contribute-plugin.md +++ b/docs/zh/developer/contribute-plugin.md @@ -17,4 +17,4 @@ - [Source Connector 开发指南](./source-connector-development.md) - [Sink Connector 开发指南](./sink-connector-development.md) - [搭建开发环境](./setup.md) -- 外部仓库指南：[Connector-v2 贡献指南](../../../seatunnel-connectors-v2/README.zh.md) +- 外部仓库指南：[Connector-v2 贡献指南](https://github.com/apache/seatunnel/blob/dev/seatunnel-connectors-v2/README.zh.md) diff --git a/docs/zh/developer/contribute-transform-v2-guide.md b/docs/zh/developer/contribute-transform-v2-guide.md index 6647521fb3..c9ef272294 100644 --- a/docs/zh/developer/contribute-transform-v2-guide.md +++ b/docs/zh/developer/contribute-transform-v2-guide.md @@ -50,10 +50,10 @@ Transform 类改动如果能同时带上代码、文档和示例，通常更容 2. [Transforms 目录](../transforms) 3. [Transform 通用参数](../transforms/common-options/common-options.md) 4. [核心 API 设计](../architecture/core-api-design.md) -5. [seatunnel-transforms-v2 README](../../../seatunnel-transforms-v2/README.zh.md) +5. [seatunnel-transforms-v2 README](https://github.com/apache/seatunnel/blob/dev/seatunnel-transforms-v2/README.zh.md) ## 什么时候继续看仓库级指南 如果你需要的是 `seatunnel-transforms-v2` 模块内部的约定、示例或更贴近源码目录的说明，再继续看仓库里的指南： -- [Transform-V2 贡献指南](../../../seatunnel-transforms-v2/README.zh.md) +- [Transform-V2 贡献指南](https://github.com/apache/seatunnel/blob/dev/seatunnel-transforms-v2/README.zh.md) diff --git a/docs/zh/developer/how-to-create-your-connector.md b/docs/zh/developer/how-to-create-your-connector.md index 17ee348330..b095ea2d07 100644 --- a/docs/zh/developer/how-to-create-your-connector.md +++ b/docs/zh/developer/how-to-create-your-connector.md @@ -1,6 +1,6 @@ # 开发自己的Connector -如果你想针对SeaTunnel新的连接器API开发自己的连接器（Connector V2），请查看[这里](../../../seatunnel-connectors-v2/README.zh.md) 。 +如果你想针对SeaTunnel新的连接器API开发自己的连接器（Connector V2），请查看[这里](https://github.com/apache/seatunnel/blob/dev/seatunnel-connectors-v2/README.zh.md) 。 ## 从这里开始 diff --git a/docs/zh/faq.md b/docs/zh/faq.md index de83e0a859..d8cb49850e 100644 --- a/docs/zh/faq.md +++ b/docs/zh/faq.md @@ -136,4 +136,4 @@ SeaTunnel 拥有完全抽象、结构化的非常优秀的架构设计和代码 ## 如果想开发自己的 source、sink、transform 时，是否需要了解 SeaTunnel 所有源代码？ 不需要，您只需要关注 source、sink、transform 对应的接口即可。 -如果你想针对 SeaTunnel API 开发自己的连接器（Connector V2），请查看**[Connector Development Guide](../../seatunnel-connectors-v2/README.zh.md)** 。 +如果你想针对 SeaTunnel API 开发自己的连接器（Connector V2），请查看**[Connector Development Guide](https://github.com/apache/seatunnel/blob/dev/seatunnel-connectors-v2/README.zh.md)** 。
