This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/dev/pr-11705-f6bfc4e07c5d319563946ecb40a18eec162d9efa in repository https://gitbox.apache.org/repos/asf/seatunnel.git
commit c367c4c8cce611ffd0939fcdf725c1315a455c6e Author: Daniel Carter <[email protected]> AuthorDate: Thu Oct 29 10:00:34 2026 +0000 [Docs][Connector-V2] Improve Paimon StarRocks and RabbitMQ connector docs (#11705) Co-authored-by: DanielCarter-stack <[email protected]> Co-authored-by: DanielCarter-stack <[email protected]> Co-authored-by: davidzollo <[email protected]> --- docs/en/connectors/sink/Paimon.md | 34 +++++----- docs/en/connectors/sink/Rabbitmq.md | 86 ++++++++++++------------- docs/en/connectors/sink/StarRocks.md | 38 +++++------ docs/en/connectors/source/Paimon.md | 71 ++++++++++++++------ docs/en/connectors/source/Rabbitmq.md | 114 ++++++++++++++++----------------- docs/en/connectors/source/StarRocks.md | 16 ++--- docs/zh/connectors/sink/Rabbitmq.md | 52 +++++++-------- docs/zh/connectors/sink/StarRocks.md | 30 ++++----- docs/zh/connectors/source/Paimon.md | 32 ++++----- docs/zh/connectors/source/StarRocks.md | 8 +-- 10 files changed, 255 insertions(+), 226 deletions(-) diff --git a/docs/en/connectors/sink/Paimon.md b/docs/en/connectors/sink/Paimon.md index 75e9b9fa71..e27ccd0300 100644 --- a/docs/en/connectors/sink/Paimon.md +++ b/docs/en/connectors/sink/Paimon.md @@ -70,26 +70,26 @@ libfb303-xxx.jar | name | type | required | default value | Description | |------------------------------|---------|----------|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| warehouse | String | Yes | - | Paimon warehouse path. Must not be blank. | -| catalog_name | String | No | paimon | The name of Paimon catalog | -| catalog_type | String | No | filesystem | Catalog type of Paimon, support filesystem and hive | +| warehouse | String | Yes | - | Paimon warehouse path. Must not be blank. | +| catalog_name | String | No | paimon | The name of the Paimon catalog. | +| catalog_type | String | No | filesystem | Catalog type of Paimon. Supports `filesystem` and `hive`. | | catalog_uri | String | Yes when `catalog_type` is `hive` | - | Catalog URI of Paimon. Required when `catalog_type` is `hive`. | -| database | String | Yes | - | The database you want to access. Must not be blank. | -| table | String | Yes | - | The table you want to access. Must not be blank. | -| user | String | No | - | Paimon user to access table | -| password | String | No | - | Paimon user password to access table | -| hdfs_site_path | String | No | - | Deprecated. The path of hdfs-site.xml. Prefer `paimon.hadoop.conf` or `paimon.hadoop.conf-path` for new jobs | -| schema_save_mode | Enum | No | CREATE_SCHEMA_WHEN_NOT_EXIST | The schema save mode | -| data_save_mode | Enum | No | APPEND_DATA | The data save mode | -| paimon.table.primary-keys | String | No | - | Default comma-separated list of columns (primary key) that identify a row in tables.(Notice: The partition field needs to be included in the primary key fields) | +| database | String | Yes | - | The database you want to access. Must not be blank. | +| table | String | Yes | - | The table you want to access. Must not be blank. | +| user | String | No | - | The Paimon user used to access the table. Required when the underlying catalog enforces authentication. | +| password | String | No | - | The Paimon user password. Required when `user` is configured. | +| hdfs_site_path | String | No | - | Deprecated. The path of `hdfs-site.xml`. Prefer `paimon.hadoop.conf` or `paimon.hadoop.conf-path` for new jobs. | +| schema_save_mode | Enum | No | CREATE_SCHEMA_WHEN_NOT_EXIST | The schema save mode. | +| data_save_mode | Enum | No | APPEND_DATA | The data save mode. | +| paimon.table.primary-keys | String | No | - | Default comma-separated list of columns (primary key) that identify a row in tables. (Notice: The partition field needs to be included in the primary key fields.) | | paimon.table.partition-keys | String | No | - | Default comma-separated list of partition fields to use when creating tables. | -| paimon.table.write-props | Map | No | - | Properties passed through to paimon table initialization, [reference](https://paimon.apache.org/docs/master/maintenance/configurations/#coreoptions). | +| paimon.table.write-props | Map | No | - | Properties passed through to Paimon table initialization, see [Paimon CoreOptions](https://paimon.apache.org/docs/master/maintenance/configurations/#coreoptions). | | table_options | Map | No | - | Sink-specific table options for SaveMode auto-create only (not runtime write-props). See below. | -| paimon.hadoop.conf | Map | No | - | Properties in hadoop conf | -| paimon.hadoop.conf-path | String | No | - | The specified loading path for the 'core-site.xml', 'hdfs-site.xml', 'hive-site.xml' files | -| paimon.table.non-primary-key | Boolean | No | false | Switch to create `table with PK` or `table without PK`. true : `table without PK`, false : `table with PK` | -| branch | String | No | - | The branch name of Paimon table to write data to. If omitted, data is written to the main branch. For non-main branches, the main table and target branch must already exist, and `schema_save_mode=RECREATE_SCHEMA` or `data_save_mode=DROP_DATA` is not supported. | -| multi_table_sink_replica | int | No | 1 | The replica number of multi table sink writer. Use this when one upstream job writes to multiple Paimon tables and each table needs more than one sink writer. | +| paimon.hadoop.conf | Map | No | - | Properties in Hadoop conf. | +| paimon.hadoop.conf-path | String | No | - | The specified loading path for `core-site.xml`, `hdfs-site.xml`, and `hive-site.xml` files. | +| paimon.table.non-primary-key | Boolean | No | false | Switch to create `table with PK` or `table without PK`. `true`: table without PK, `false`: table with PK. | +| branch | String | No | - | The branch name of the Paimon table to write data to. If omitted, data is written to the main branch. For non-main branches, the main table and target branch must already exist, and `schema_save_mode=RECREATE_SCHEMA` or `data_save_mode=DROP_DATA` is not supported. | +| multi_table_sink_replica | int | No | 1 | The replica number of multi table sink writer. Use this when one upstream job writes to multiple Paimon tables and each table needs more than one sink writer. | ### table_options [Map] diff --git a/docs/en/connectors/sink/Rabbitmq.md b/docs/en/connectors/sink/Rabbitmq.md index f2cfa74cab..2a9bd78e0b 100644 --- a/docs/en/connectors/sink/Rabbitmq.md +++ b/docs/en/connectors/sink/Rabbitmq.md @@ -21,58 +21,58 @@ Used to write data to RabbitMQ queues. ## Options -| name | type | required | default value | -|----------------------------|---------|----------|---------------| -| host | string | yes | - | -| port | int | yes | - | -| virtual_host | string | yes | - | -| username | string | no | - | -| password | string | no | - | -| queue_name | string | yes | - | -| format | string | no | json | -| protobuf_schema | string | no | - | -| protobuf_message_name | string | no | - | -| url | string | no | - | -| uri | string | no | - | -| ssl | boolean | no | false | -| routing_key | string | no | - | -| exchange | string | no | - | -| network_recovery_interval | int | no | - | -| topology_recovery_enabled | boolean | no | - | -| AUTOMATIC_RECOVERY_ENABLED | boolean | no | - | -| connection_timeout | int | no | - | -| rabbitmq.config | map | no | - | -| common-options | | no | - | -| durable | boolean | no | true | -| exclusive | boolean | no | false | -| auto_delete | boolean | no | false | -| passive | boolean | no | false | +| name | type | required | default value | description | +|----------------------------|---------|----------|---------------|------------------------------------------------------------------------------------------------------------------------------| +| host | string | yes | - | The default host to use for connections. | +| port | int | yes | - | The default port to use for connections. | +| virtual_host | string | yes | - | Virtual host to use when connecting to the broker. | +| username | string | no | - | The AMQP user name to use when connecting to the broker. | +| password | string | no | - | The password to use when connecting to the broker. Must be configured together with `username`. | +| queue_name | string | yes | - | The queue to write the message to. If `routing_key` is not configured, the connector publishes to this queue through the default exchange. | +| url | string | no | - | Convenience method for setting host, port, username, password and virtual host in an AMQP URI. | +| routing_key | string | no | - | Routing key used to publish messages. Configure together with `exchange` to publish through a specific exchange. | +| exchange | string | no | - | Exchange used when `routing_key` is configured. | +| network_recovery_interval | int | no | - | How long automatic recovery waits before attempting to reconnect, in milliseconds. | +| topology_recovery_enabled | boolean | no | - | Whether to enable topology recovery. | +| AUTOMATIC_RECOVERY_ENABLED | boolean | no | - | Whether to enable connection recovery. The option key is currently uppercase in the connector configuration. | +| connection_timeout | int | no | - | TCP connection establishment timeout, in milliseconds; `0` means infinite. | +| rabbitmq.config | map | no | - | Extra RabbitMQ client parameters. See the [official RabbitMQ documentation](https://www.rabbitmq.com/configure.html). | +| common-options | | no | - | Sink plugin common parameters. See [Sink Common Options](../common-options/sink-common-options.md). | +| durable | boolean | no | true | Whether the queue survives a server restart. Used when the connector declares the target queue. | +| exclusive | boolean | no | false | Whether the queue is used only by the current connection and is deleted when the connection closes. | +| auto_delete | boolean | no | false | Whether the queue is deleted automatically when the last consumer unsubscribes. | +| format | string | no | json | The message payload format: `json` or `protobuf`. | +| protobuf_schema | string | no | - | Protobuf schema used when `format` is `protobuf`. | +| protobuf_message_name | string | no | - | Protobuf message name used when `format` is `protobuf`. | +| uri | string | no | - | Legacy alias for `url`; configure only one. | +| ssl | boolean | no | false | Enable SSL/TLS for host-and-port connections. | +| passive | boolean | no | false | Check that the configured queue exists without declaring it. | ### host [string] -the default host to use for connections +The default host to use for connections. ### port [int] -the default port to use for connections +The default port to use for connections. ### virtual_host [string] -virtual host – the virtual host to use when connecting to the broker +Virtual host – the virtual host to use when connecting to the broker. ### username [string] -the AMQP user name to use when connecting to the broker +The AMQP user name to use when connecting to the broker. ### password [string] -the password to use when connecting to the broker +The password to use when connecting to the broker. `username` and `password` should be configured together. ### url [string] -convenience method for setting the fields in an AMQP URI: host, port, username, password and virtual host +Convenience method for setting the fields in an AMQP URI: host, port, username, password and virtual host. ### uri [string] @@ -110,21 +110,21 @@ The exchange used when `routing_key` is configured. ### network_recovery_interval [int] -how long will automatic recovery wait before attempting to reconnect, in ms +How long automatic recovery waits before attempting to reconnect, in milliseconds. ### topology_recovery_enabled [boolean] -if true, enables topology recovery +If `true`, enables topology recovery. ### AUTOMATIC_RECOVERY_ENABLED [boolean] -If true, enables connection recovery. +If `true`, enables connection recovery. The option key is currently uppercase in the connector configuration. Use `AUTOMATIC_RECOVERY_ENABLED`, not `automatic_recovery_enabled`. ### connection_timeout [int] -connection TCP establishment timeout in milliseconds; zero for infinite +TCP connection establishment timeout in milliseconds; `0` means infinite. ### rabbitmq.config [map] @@ -134,22 +134,22 @@ Note: the sink currently parses this option but does not apply its entries to th ### common options -Sink plugin common parameters, please refer to [Sink Common Options](../common-options/sink-common-options.md) for details +Sink plugin common parameters. See [Sink Common Options](../common-options/sink-common-options.md) for details. ### durable -- true: The queue will survive on server restart. -- false: The queue will be deleted on server restart. +- `true`: The queue will survive on server restart. +- `false`: The queue will be deleted on server restart. ### exclusive -- true: The queue is used only by the current connection and will be deleted when the connection closes. -- false: The queue can be used by multiple connections. +- `true`: The queue is used only by the current connection and will be deleted when the connection closes. +- `false`: The queue can be used by multiple connections. ### auto_delete -- true: The queue will be deleted automatically when the last consumer unsubscribes. -- false: The queue will not be automatically deleted. +- `true`: The queue will be deleted automatically when the last consumer unsubscribes. +- `false`: The queue will not be automatically deleted. ### passive diff --git a/docs/en/connectors/sink/StarRocks.md b/docs/en/connectors/sink/StarRocks.md index 4f37e22152..f86f0b0946 100644 --- a/docs/en/connectors/sink/StarRocks.md +++ b/docs/en/connectors/sink/StarRocks.md @@ -161,7 +161,7 @@ sink { base-url = "jdbc:mysql://127.0.0.1:9030" nodeUrls = ["127.0.0.1:8030"] username = "root" - password = "" + password = "secret" database = "test" schema_save_mode = "CREATE_SCHEMA_WHEN_NOT_EXIST" table_options = { @@ -205,8 +205,8 @@ sink { StarRocks { nodeUrls = ["starrocks-fe:8030"] base-url = "jdbc:mysql://starrocks-fe:9030/mydb" - username = root - password = "" + username = "root" + password = "secret" database = "mydb" table = "mytable" batch_max_rows = 10000 @@ -284,8 +284,8 @@ source { sink { StarRocks { nodeUrls = ["e2e_starRocksdb:8030"] - username = root - password = "" + username = "root" + password = "secret" database = "test" table = "e2e_table_sink" batch_max_rows = 10 @@ -303,8 +303,8 @@ sink { sink { StarRocks { nodeUrls = ["e2e_starRocksdb:8030"] - username = root - password = "" + username = "root" + password = "secret" database = "test" table = "e2e_table_sink" ... @@ -322,8 +322,8 @@ sink { StarRocks { nodeUrls = ["e2e_starRocksdb:8030"] base-url = "jdbc:mysql://e2e_starRocksdb:9030/" - username = root - password = "" + username = "root" + password = "secret" database = "test" table = "e2e_table_sink" batch_max_rows = 10 @@ -342,8 +342,8 @@ sink { StarRocks { nodeUrls = ["e2e_starRocksdb:8030"] base-url = "jdbc:mysql://e2e_starRocksdb:9030/" - username = root - password = "" + username = "root" + password = "secret" database = "test" table = "e2e_table_sink" batch_max_rows = 10 @@ -363,8 +363,8 @@ sink { StarRocks { nodeUrls = ["e2e_starRocksdb:8030"] base-url = "jdbc:mysql://e2e_starRocksdb:9030/" - username = root - password = "" + username = "root" + password = "secret" database = "test" table = "test_${schema_name}_${table_name}" schema_save_mode = "CREATE_SCHEMA_WHEN_NOT_EXIST" @@ -504,8 +504,8 @@ sink { StarRocks { nodeUrls = ["e2e_starRocksdb:8030"] base-url = "jdbc:mysql://e2e_starRocksdb:9030/" - username = root - password = "" + username = "root" + password = "secret" database = "${database_name}_test" table = "${table_name}_test" ... @@ -549,8 +549,8 @@ sink { StarRocks { nodeUrls = ["e2e_starRocksdb:8030"] base-url = "jdbc:mysql://e2e_starRocksdb:9030/" - username = root - password = "" + username = "root" + password = "secret" database = "${schema_name}_test" table = "${table_name}_test" ... @@ -589,8 +589,8 @@ sink { StarRocks { nodeUrls = ["starrocks-fe:8030"] base-url = "jdbc:mysql://starrocks-fe:9030/" - username = root - password = "" + username = "root" + password = "secret" database = "mydb" table = "mytable" labelPrefix = "unique-job-label" diff --git a/docs/en/connectors/source/Paimon.md b/docs/en/connectors/source/Paimon.md index a9a7bccc35..3b37e7a547 100644 --- a/docs/en/connectors/source/Paimon.md +++ b/docs/en/connectors/source/Paimon.md @@ -52,37 +52,41 @@ Read data from Apache Paimon. ## Options -| name | type | required | default value | -|-------------------------|----------|----------------|---------------| -| warehouse | String | Yes | - | -| catalog_name | String | No | paimon | -| catalog_type | String | No | filesystem | -| catalog_uri | String | Yes when `catalog_type` is `hive` | - | -| database | String | Yes | - | -| table | String | Yes when `table_list` is absent | - | -| table_list | array | Yes when `table` is absent | - | -| user | String | No | - | -| password | String | No | - | -| hdfs_site_path | String | No | - | -| query | String | No | - | -| paimon.hadoop.conf | Map | No | - | -| paimon.hadoop.conf-path | String | No | - | +| name | type | required | default value | description | +|-------------------------|----------|----------------|---------------|------------------------------------------------------------------------------------------------------------------------------| +| warehouse | String | Yes | - | Paimon warehouse path. | +| catalog_name | String | No | paimon | The name of the Paimon catalog. | +| catalog_type | String | No | filesystem | Catalog type of Paimon, supports `filesystem` and `hive`. | +| catalog_uri | String | Yes when `catalog_type` is `hive` | - | Catalog URI of Paimon. Required when `catalog_type` is `hive`. | +| database | String | Yes | - | The database you want to access. | +| table | String | Yes when `table_list` is absent | - | The table you want to access. Configure exactly one of `table` and `table_list`. | +| table_list | array | Yes when `table` is absent | - | The list of tables to read. Each item must contain `database` and `table`, and may contain its own `query`. | +| user | String | No | - | The Paimon user used to access the table (for example, with a `hive` catalog that enables authentication). | +| password | String | No | - | The Paimon user password. Required when `user` is configured. | +| hdfs_site_path | String | No | - | The file path of `hdfs-site.xml`. Deprecated; prefer `paimon.hadoop.conf` or `paimon.hadoop.conf-path` for new jobs. | +| query | String | No | - | The filter condition applied to the table read. If not specified, all rows are read. | +| paimon.hadoop.conf | Map | No | - | Properties applied to the Hadoop configuration. | +| paimon.hadoop.conf-path | String | No | - | The loading path for `core-site.xml`, `hdfs-site.xml`, and `hive-site.xml` files. | ### warehouse [string] Paimon warehouse path. Must not be blank. +### catalog_name [string] + +The name of the Paimon catalog. Default value is `paimon`. + ### catalog_type [string] -Catalog type of Paimon, support filesystem and hive +Catalog type of Paimon. Supports `filesystem` and `hive`. Default value is `filesystem`. ### catalog_uri [string] -Catalog URI of Paimon. This option is required when `catalog_type` is `hive`. +Catalog URI of Paimon. This option is required when `catalog_type` is `hive` (for example, `thrift://hadoop04:9083`). ### database [string] -The database you want to access +The database you want to access. ### table [string] @@ -90,7 +94,29 @@ The table you want to access. Configure exactly one of `table` and `table_list`. ### table_list [array] -The list of tables to read. Configure exactly one of `table` and `table_list`. Each item must contain `table`, and can contain its own `query`. +The list of tables to read. Configure exactly one of `table` and `table_list`. Each item must contain `database` and `table`, and may contain its own `query` filter. + +```hocon +table_list = [ + { + database = "default" + table = "table1" + query = "select * from table1 where id > 100" + }, + { + database = "default" + table = "table2" + } +] +``` + +### user [string] + +The Paimon user used to access the table. Use it together with `password`. This is mainly required when the underlying catalog enforces authentication. + +### password [string] + +The Paimon user password. Required when `user` is configured. ### hdfs_site_path [string] @@ -159,10 +185,12 @@ source { database = "default" table_list = [ { + database = "default" table = "table1" query = "select * from table1 where id > 100" }, { + database = "default" table = "table2" query = "select * from table2 where id > 100" } @@ -278,13 +306,14 @@ If you want to read the changelog of the Paimon table, first set the `changelog- ### Note -Currently, batch reads are always the latest snapshot read, so to read full changelog data, you need to use stream reads and start stream reads before writing data to the Paimon table, and to ensure order, the parallelism of the stream read task should be set to 1. +Currently, batch reads are always the latest snapshot read, so to read full changelog data, you need to use stream reads and start stream reads before writing data to the Paimon table. To ensure ordering, the parallelism of the stream read task should be set to 1. ### Streaming read example + ```hocon env { parallelism = 1 - job.mode = "Streaming" + job.mode = "STREAMING" } source { diff --git a/docs/en/connectors/source/Rabbitmq.md b/docs/en/connectors/source/Rabbitmq.md index 1484dc7067..4f615a3044 100644 --- a/docs/en/connectors/source/Rabbitmq.md +++ b/docs/en/connectors/source/Rabbitmq.md @@ -45,65 +45,65 @@ This source supports connection/authentication and existing-queue metadata check ## Options -| name | type | required | default value | -| -------------------------- | ------- | -------- | ------------- | -| host | string | yes | - | -| port | int | yes | - | -| virtual_host | string | no | - | -| username | string | no | - | -| password | string | no | - | -| queue_name | string | no | - | -| schema | config | no | - | -| tables_configs | array | no | - | -| format | string | no | json | -| protobuf_schema | string | no | - | -| protobuf_message_name | string | no | - | -| url | string | no | - | -| uri | string | no | - | -| ssl | boolean | no | false | -| routing_key | string | no | - | -| exchange | string | no | - | -| network_recovery_interval | int | no | - | -| topology_recovery_enabled | boolean | no | - | -| AUTOMATIC_RECOVERY_ENABLED | boolean | no | - | -| connection_timeout | int | no | - | -| requested_channel_max | int | no | - | -| requested_frame_max | int | no | - | -| requested_heartbeat | int | no | - | -| prefetch_count | int | no | - | -| delivery_timeout | int | no | - | -| use_correlation_id | boolean | no | - | -| common-options | | no | - | -| durable | boolean | no | true | -| exclusive | boolean | no | false | -| auto_delete | boolean | no | false | -| passive | boolean | no | false | +| name | type | required | default value | description | +| -------------------------- | ------- | -------- | ------------- |------------------------------------------------------------------------------------------------------------------------------| +| host | string | yes | - | The default host to use for connections. | +| port | int | yes | - | The default port to use for connections. | +| virtual_host | string | no | - | Virtual host to use when connecting to the broker. | +| username | string | no | - | The AMQP user name to use when connecting to the broker. | +| password | string | no | - | The password to use when connecting to the broker. Must be configured together with `username`. | +| queue_name | string | no | - | The queue to consume messages from. Required if `tables_configs` is not configured. | +| schema | config | no | - | The schema of upstream data. Required if `tables_configs` is not configured. See [Schema Feature](../../introduction/concepts/schema-feature.md). | +| tables_configs | array | no | - | Multi-queue configuration. Each item must contain `queue_name` and `schema`. Cannot be configured together with root-level `queue_name`. | +| url | string | no | - | Convenience method for setting host, port, username, password and virtual host in an AMQP URI. | +| routing_key | string | no | - | Optional RabbitMQ routing key inherited from the shared RabbitMQ configuration. Not required for normal queue consumption. | +| exchange | string | no | - | Optional RabbitMQ exchange inherited from the shared RabbitMQ configuration. Not required for normal queue consumption. | +| network_recovery_interval | int | no | - | How long automatic recovery waits before attempting to reconnect, in milliseconds. | +| topology_recovery_enabled | boolean | no | - | Whether to enable topology recovery. | +| AUTOMATIC_RECOVERY_ENABLED | boolean | no | - | Whether to enable connection recovery. The option key is currently uppercase in the connector configuration. | +| connection_timeout | int | no | - | TCP connection establishment timeout, in milliseconds; `0` means infinite. | +| requested_channel_max | int | no | - | Initially requested maximum channel number; `0` means unlimited. Must be between 0 and 65535. | +| requested_frame_max | int | no | - | The requested maximum frame size. | +| requested_heartbeat | int | no | - | The requested heartbeat timeout. Must be between 0 and 65535. | +| prefetch_count | int | no | - | The maximum number of messages to receive without acknowledgement. | +| delivery_timeout | int | no | - | Maximum wait time, in milliseconds, for the next message delivery. | +| use_correlation_id | boolean | no | - | Whether the consumed messages provide a unique correlation id that can be used to deduplicate messages when acknowledgments fail. | +| common-options | | no | - | Source plugin common parameters. See [Source Common Options](../common-options/source-common-options.md). | +| durable | boolean | no | true | Whether the queue survives a server restart. | +| exclusive | boolean | no | false | Whether the queue is used only by the current connection and is deleted when the connection closes. | +| auto_delete | boolean | no | false | Whether the queue is deleted automatically when the last consumer unsubscribes. | +| format | string | no | json | The message payload format: `json` or `protobuf`. | +| protobuf_schema | string | no | - | Protobuf schema used when `format` is `protobuf`. | +| protobuf_message_name | string | no | - | Protobuf message name used when `format` is `protobuf`. | +| uri | string | no | - | Legacy alias for `url`; configure only one. | +| ssl | boolean | no | false | Enable SSL/TLS for host-and-port connections. | +| passive | boolean | no | false | Check that the configured queue exists without declaring it. | ### host [string] -the default host to use for connections +The default host to use for connections. ### port [int] -the default port to use for connections +The default port to use for connections. ### virtual_host [string] -virtual host – the virtual host to use when connecting to the broker +Virtual host – the virtual host to use when connecting to the broker. ### username [string] -the AMQP user name to use when connecting to the broker +The AMQP user name to use when connecting to the broker. ### password [string] -the password to use when connecting to the broker +The password to use when connecting to the broker. `username` and `password` should be configured together. ### url [string] -convenience method for setting the fields in an AMQP URI: host, port, username, password and virtual host +Convenience method for setting the fields in an AMQP URI: host, port, username, password and virtual host. ### uri [string] @@ -117,7 +117,7 @@ When `url` uses an `amqps://` URI, the broker certificate is verified against th ### queue_name [string] -the queue to consume messages from. *Note: Required if `tables_configs` is not configured.* +The queue to consume messages from. *Note: Required if `tables_configs` is not configured.* ### routing_key [string] @@ -131,7 +131,7 @@ Optional RabbitMQ exchange inherited from the shared RabbitMQ configuration. It #### fields [Config] -the schema fields of upstream data. For more details, please refer to [Schema Feature](../../introduction/concepts/schema-feature.md). *Note: Required if `tables_configs` is not configured.* +The schema fields of upstream data. For more details, please refer to [Schema Feature](../../introduction/concepts/schema-feature.md). *Note: Required if `tables_configs` is not configured.* ### tables_configs [array] @@ -151,43 +151,43 @@ Effective when `format` is `protobuf`. Specifies the Protobuf message name to de ### network_recovery_interval [int] -how long will automatic recovery wait before attempting to reconnect, in ms +How long automatic recovery waits before attempting to reconnect, in milliseconds. ### topology_recovery_enabled [boolean] -if true, enables topology recovery +If `true`, enables topology recovery. ### AUTOMATIC_RECOVERY_ENABLED [boolean] -If true, enables connection recovery. +If `true`, enables connection recovery. The option key is currently uppercase in the connector configuration. Use `AUTOMATIC_RECOVERY_ENABLED`, not `automatic_recovery_enabled`. ### connection_timeout [int] -connection tcp establishment timeout in milliseconds; zero for infinite +TCP connection establishment timeout in milliseconds; `0` means infinite. ### requested_channel_max [int] -initially requested maximum channel number; zero for unlimited +Initially requested maximum channel number; `0` means unlimited. **Note:** The value must be between 0 and 65535 (unsigned short in AMQP 0-9-1). ### requested_frame_max [int] -the requested maximum frame size +The requested maximum frame size. ### requested_heartbeat [int] -Set the requested heartbeat timeout +The requested heartbeat timeout. **Note:** The value must be between 0 and 65535 (unsigned short in AMQP 0-9-1). ### prefetch_count [int] -prefetchCount the max number of messages to receive without acknowledgement +The maximum number of messages to receive without acknowledgement. ### delivery_timeout [int] -deliveryTimeout maximum wait time, in milliseconds, for the next message delivery +Maximum wait time, in milliseconds, for the next message delivery. ### use_correlation_id [boolean] @@ -195,22 +195,22 @@ Whether the consumed messages provide a unique correlation id that can be used t ### common options -Source plugin common parameters, please refer to [Source Common Options](../common-options/source-common-options.md) for details +Source plugin common parameters. See [Source Common Options](../common-options/source-common-options.md) for details. ### durable -- true: The queue will survive on server restart. -- false: The queue will be deleted on server restart. +- `true`: The queue will survive on server restart. +- `false`: The queue will be deleted on server restart. ### exclusive -- true: The queue is used only by the current connection and will be deleted when the connection closes. -- false: The queue can be used by multiple connections. +- `true`: The queue is used only by the current connection and will be deleted when the connection closes. +- `false`: The queue can be used by multiple connections. ### auto_delete -- true: The queue will be deleted automatically when the last consumer unsubscribes. -- false: The queue will not be automatically deleted. +- `true`: The queue will be deleted automatically when the last consumer unsubscribes. +- `false`: The queue will not be automatically deleted. ### passive diff --git a/docs/en/connectors/source/StarRocks.md b/docs/en/connectors/source/StarRocks.md index 6c5a9a95e4..f46d9e5da0 100644 --- a/docs/en/connectors/source/StarRocks.md +++ b/docs/en/connectors/source/StarRocks.md @@ -31,7 +31,7 @@ delivers the query plan as a parameter to BE nodes, and then obtains data result | table_list | array | no | - | Tables to read. Required when `table` is not configured. Each entry can define its own `schema` and filter. | | schema | config | no | - | Output schema. Configure it at the top level for `table`, or inside each `table_list` entry for multi-table reads. | | scan_filter | string | no | "" | Source-side filter expression passed to StarRocks. | -| request_tablet_size | int | no | Integer.MAX_VALUE | Maximum tablets in one SeaTunnel split. Smaller values can create more splits. | +| request_tablet_size | int | no | Integer.MAX_VALUE | Maximum tablets in one SeaTunnel split. Smaller values create more splits. | | scan_connect_timeout_ms | int | no | 1000 | Timeout in milliseconds when connecting to StarRocks BE for scan. | | scan_query_timeout_sec | int | no | 3600 | Query timeout in seconds. `-1` means no timeout. | | scan_keep_alive_min | int | no | 10 | Keep-alive time of the query task, in minutes. | @@ -46,15 +46,15 @@ delivers the query plan as a parameter to BE nodes, and then obtains data result ### username [string] -`StarRocks` user username +`StarRocks` user username. ### password [string] -`StarRocks` user password +`StarRocks` user password. ### database [string] -The name of StarRocks database +The name of StarRocks database. ### table [string] @@ -157,8 +157,8 @@ The parameter of the scan data from be source { StarRocks { nodeUrls = ["starrocks_e2e:8030"] - username = root - password = "" + username = "root" + password = "secret" database = "test" table = "e2e_table_source" scan_batch_rows = 10 @@ -193,8 +193,8 @@ source { source { StarRocks { nodeUrls = ["starrocks_e2e:8030"] - username = root - password = "" + username = "root" + password = "secret" database = "test" table_list = [ { diff --git a/docs/zh/connectors/sink/Rabbitmq.md b/docs/zh/connectors/sink/Rabbitmq.md index 9221c00d5f..b6600b65ab 100644 --- a/docs/zh/connectors/sink/Rabbitmq.md +++ b/docs/zh/connectors/sink/Rabbitmq.md @@ -21,32 +21,32 @@ import ChangeLog from '../changelog/connector-rabbitmq.md'; ## 接收器选项 -| 名称 | 类型 | 是否必须 | 默认值 | -|----------------------------|---------|------|-------| -| host | string | 是 | - | -| port | int | 是 | - | -| virtual_host | string | 是 | - | -| username | string | 否 | - | -| password | string | 否 | - | -| queue_name | string | 是 | - | -| format | string | 否 | json | -| protobuf_schema | string | 否 | - | -| protobuf_message_name | string | 否 | - | -| url | string | 否 | - | -| uri | string | 否 | - | -| ssl | boolean | 否 | false | -| routing_key | string | 否 | - | -| exchange | string | 否 | - | -| network_recovery_interval | int | 否 | - | -| topology_recovery_enabled | boolean | 否 | - | -| AUTOMATIC_RECOVERY_ENABLED | boolean | 否 | - | -| connection_timeout | int | 否 | - | -| rabbitmq.config | map | 否 | - | -| durable | boolean | 否 | true | -| exclusive | boolean | 否 | false | -| auto_delete | boolean | 否 | false | -| passive | boolean | 否 | false | -| common-options | | 否 | - | +| 名称 | 类型 | 是否必须 | 默认值 | 描述 | +|----------------------------|---------|------|-------|---| +| host | string | 是 | - | RabbitMQ 服务器地址。 | +| port | int | 是 | - | RabbitMQ 服务器端口。 | +| virtual_host | string | 是 | - | 连接 Broker 时使用的 virtual host。 | +| username | string | 否 | - | 连接 Broker 时使用的用户名。 | +| password | string | 否 | - | 连接时使用的密码；需要与 `username` 一起配置。 | +| queue_name | string | 是 | - | 目标队列名；未配置 `routing_key` 时通过默认 exchange 写入。 | +| format | string | 否 | json | 消息体格式，支持 `json` 和 `protobuf`。 | +| protobuf_schema | string | 否 | - | `format` 为 `protobuf` 时使用的 Protobuf Schema。 | +| protobuf_message_name | string | 否 | - | `format` 为 `protobuf` 时使用的 Protobuf 消息名称。 | +| url | string | 否 | - | 使用 AMQP URI 配置主机、端口、用户名、密码和 virtual host。 | +| uri | string | 否 | - | `url` 的兼容别名，两者只能配置一个。 | +| ssl | boolean | 否 | false | 通过主机和端口连接时是否启用 SSL/TLS。 | +| routing_key | string | 否 | - | 发送消息使用的 routing key，与 `exchange` 一起配置。 | +| exchange | string | 否 | - | 配置 `routing_key` 时使用的 exchange。 | +| network_recovery_interval | int | 否 | - | 自动恢复连接的重试间隔，单位为毫秒。 | +| topology_recovery_enabled | boolean | 否 | - | 是否启用拓扑恢复。 | +| AUTOMATIC_RECOVERY_ENABLED | boolean | 否 | - | 是否启用连接恢复；配置键使用大写。 | +| connection_timeout | int | 否 | - | TCP 建连超时，单位为毫秒；`0` 表示无限等待。 | +| rabbitmq.config | map | 否 | - | 额外的 RabbitMQ 客户端配置。 | +| durable | boolean | 否 | true | 是否声明持久化队列，在服务器重启后保留。 | +| exclusive | boolean | 否 | false | 是否声明仅当前连接使用、连接关闭后删除的队列。 | +| auto_delete | boolean | 否 | false | 是否在最后一个消费者取消订阅后自动删除队列。 | +| passive | boolean | 否 | false | 是否只检查队列已存在，不主动声明队列。 | +| common-options | | 否 | - | Sink 插件通用配置，参见 [Sink 通用选项](../common-options/sink-common-options.md)。 | ### host [string] diff --git a/docs/zh/connectors/sink/StarRocks.md b/docs/zh/connectors/sink/StarRocks.md index ba69a2d3c9..24e1e36b2e 100644 --- a/docs/zh/connectors/sink/StarRocks.md +++ b/docs/zh/connectors/sink/StarRocks.md @@ -156,7 +156,7 @@ sink { base-url = "jdbc:mysql://127.0.0.1:9030" nodeUrls = ["127.0.0.1:8030"] username = "root" - password = "" + password = "secret" database = "test" schema_save_mode = "CREATE_SCHEMA_WHEN_NOT_EXIST" table_options = { @@ -200,8 +200,8 @@ sink { StarRocks { nodeUrls = ["starrocks-fe:8030"] base-url = "jdbc:mysql://starrocks-fe:9030/mydb" - username = root - password = "" + username = "root" + password = "secret" database = "mydb" table = "mytable" batch_max_rows = 10000 @@ -280,8 +280,8 @@ sink { StarRocks { nodeUrls = ["e2e_starRocksdb:8030"] base-url = "jdbc:mysql://e2e_starRocksdb:9030/" - username = root - password = "" + username = "root" + password = "secret" database = "test" table = "e2e_table_sink" batch_max_rows = 10 @@ -300,8 +300,8 @@ sink { StarRocks { nodeUrls = ["e2e_starRocksdb:8030"] base-url = "jdbc:mysql://e2e_starRocksdb:9030/" - username = root - password = "" + username = "root" + password = "secret" database = "test" table = "e2e_table_sink" ... @@ -319,8 +319,8 @@ sink { StarRocks { nodeUrls = ["e2e_starRocksdb:8030"] base-url = "jdbc:mysql://e2e_starRocksdb:9030/" - username = root - password = "" + username = "root" + password = "secret" database = "test" table = "e2e_table_sink" batch_max_rows = 10 @@ -340,8 +340,8 @@ sink { StarRocks { nodeUrls = ["e2e_starRocksdb:8030"] base-url = "jdbc:mysql://e2e_starRocksdb:9030/" - username = root - password = "" + username = "root" + password = "secret" database = "test" table = "e2e_table_sink" batch_max_rows = 10 @@ -361,8 +361,8 @@ sink { StarRocks { nodeUrls = ["e2e_starRocksdb:8030"] base-url = "jdbc:mysql://e2e_starRocksdb:9030/" - username = root - password = "" + username = "root" + password = "secret" database = "test" table = "test_${schema_name}_${table_name}" schema_save_mode = "CREATE_SCHEMA_WHEN_NOT_EXIST" @@ -405,8 +405,8 @@ sink { StarRocks { nodeUrls = ["starrocks-fe:8030"] base-url = "jdbc:mysql://starrocks-fe:9030/" - username = root - password = "" + username = "root" + password = "secret" database = "mydb" table = "mytable" labelPrefix = "unique-job-label" diff --git a/docs/zh/connectors/source/Paimon.md b/docs/zh/connectors/source/Paimon.md index 5d2c512e67..0c4bc651c6 100644 --- a/docs/zh/connectors/source/Paimon.md +++ b/docs/zh/connectors/source/Paimon.md @@ -52,21 +52,21 @@ import ChangeLog from '../changelog/connector-paimon.md'; ## 配置选项 -| 名称 | 类型 | 是否必须 | 默认值 | -|-------------------------|----------|--------|---------------| -| warehouse | String | 是 | - | -| catalog_name | String | 否 | paimon | -| catalog_type | String | 否 | filesystem | -| catalog_uri | String | 当 `catalog_type` 为 `hive` 时是 | - | -| database | String | 是 | - | -| table | String | 未配置 `table_list` 时是 | - | -| table_list | array | 未配置 `table` 时是 | - | -| user | String | 否 | - | -| password | String | 否 | - | -| hdfs_site_path | String | 否 | - | -| query | String | 否 | - | -| paimon.hadoop.conf | Map | 否 | - | -| paimon.hadoop.conf-path | String | 否 | - | +| 名称 | 类型 | 是否必须 | 默认值 | 描述 | +|-------------------------|----------|--------|---------------|---| +| warehouse | String | 是 | - | Paimon warehouse 路径，不能为空白字符串。 | +| catalog_name | String | 否 | paimon | Paimon Catalog 名称。 | +| catalog_type | String | 否 | filesystem | Paimon Catalog 类型，支持 `filesystem` 和 `hive`。 | +| catalog_uri | String | 当 `catalog_type` 为 `hive` 时是 | - | Paimon Catalog URI；`catalog_type` 为 `hive` 时必须配置。 | +| database | String | 是 | - | 需要访问的数据库。 | +| table | String | 未配置 `table_list` 时是 | - | 需要访问的表；必须在 `table` 和 `table_list` 中选择一个配置。 | +| table_list | array | 未配置 `table` 时是 | - | 需要读取的表列表；每项必须包含 `database` 和 `table`，可单独配置 `query`。 | +| user | String | 否 | - | 访问 Paimon 表的用户名。 | +| password | String | 否 | - | 访问 Paimon 表的密码，需要与 `user` 一起配置。 | +| hdfs_site_path | String | 否 | - | 已弃用的 `hdfs-site.xml` 路径；建议使用 `paimon.hadoop.conf` 或 `paimon.hadoop.conf-path`。 | +| query | String | 否 | - | 读取表时使用的过滤查询；未配置时读取所有行。 | +| paimon.hadoop.conf | Map | 否 | - | 应用到 Hadoop 配置的属性。 | +| paimon.hadoop.conf-path | String | 否 | - | `core-site.xml`、`hdfs-site.xml` 和 `hive-site.xml` 的加载路径。 | ### warehouse [string] @@ -290,7 +290,7 @@ source { ```hocon env { parallelism = 1 - job.mode = "Streaming" + job.mode = "STREAMING" } source { diff --git a/docs/zh/connectors/source/StarRocks.md b/docs/zh/connectors/source/StarRocks.md index 38a3a69c32..cb0eb54872 100644 --- a/docs/zh/connectors/source/StarRocks.md +++ b/docs/zh/connectors/source/StarRocks.md @@ -155,8 +155,8 @@ partition[5] 从 be_node_3 读取 tablet 数据：tablet[14,15] source { StarRocks { nodeUrls = ["starrocks_e2e:8030"] - username = root - password = "" + username = "root" + password = "secret" database = "test" table = "e2e_table_source" scan_batch_rows = 10 @@ -191,8 +191,8 @@ source { source { StarRocks { nodeUrls = ["starrocks_e2e:8030"] - username = root - password = "" + username = "root" + password = "secret" database = "test" table_list = [ {
