Re: [PR] [Fix][Connector-V2] Fixed adding table comments [seatunnel]

2025-01-14 Thread via GitHub
Hisoka-X commented on PR #8514: URL: https://github.com/apache/seatunnel/pull/8514#issuecomment-2589415493 Please update the docs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [I] seatunnel-web也应该来个docker的 [seatunnel]

2025-01-14 Thread via GitHub
cobolbaby commented on issue #7850: URL: https://github.com/apache/seatunnel/issues/7850#issuecomment-2589692070 Although seatunnel-web provides a Docker build script, it would be even better if the official team could provide image maintenance. -- This is an automated message from the Ap

Re: [I] [Feature][Plugin] Support Datalake delta [seatunnel]

2025-01-14 Thread via GitHub
cobolbaby commented on issue #1030: URL: https://github.com/apache/seatunnel/issues/1030#issuecomment-2589707635 @liunaijie I see you marked this as completed. Do you have any details on the Delta Lake support? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [Fix][Connector-V2] Fixed adding table comments [seatunnel]

2025-01-14 Thread via GitHub
hailin0 commented on PR #8514: URL: https://github.com/apache/seatunnel/pull/8514#issuecomment-2589715505 > Please update the docs. https://github.com/apache/seatunnel/blob/dev/docs/en/concept/sink-options-placeholders.md https://github.com/apache/seatunnel/blob/dev/docs/zh/concep

Re: [I] SeaTunnel community meeting Topic collect [seatunnel]

2025-01-14 Thread via GitHub
hailin0 closed issue #8517: SeaTunnel community meeting Topic collect URL: https://github.com/apache/seatunnel/issues/8517 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[I] 数据同步速度主键下降/ The speed of data transfer decrease gradually [seatunnel]

2025-01-14 Thread via GitHub
Seeumt opened a new issue, #8521: URL: https://github.com/apache/seatunnel/issues/8521 **情况概述:** source: 达梦数据库 sink:starrocks 表数据量大约900万行,35个字段 同步速度刚开始8000条/秒

Re: [I] 数据同步速度主键下降/ The speed of data transfer decrease gradually [seatunnel]

2025-01-14 Thread via GitHub
davidzollo closed issue #8521: 数据同步速度主键下降/ The speed of data transfer decrease gradually URL: https://github.com/apache/seatunnel/issues/8521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [I] 数据同步速度主键下降/ The speed of data transfer decrease gradually [seatunnel]

2025-01-14 Thread via GitHub
davidzollo commented on issue #8521: URL: https://github.com/apache/seatunnel/issues/8521#issuecomment-2589451775 After analyzing the runtime log, it seems that the JVM memory is too small, with only 2 GB allocated. The logs indicate frequent GC activity, and the CPU load has reached 4.65.

Re: [PR] [Improve][Connector-file-base] In large file scenarios, split the single file into multiple shards [seatunnel]

2025-01-14 Thread via GitHub
sohurdc commented on code in PR #8507: URL: https://github.com/apache/seatunnel/pull/8507#discussion_r1914816014 ## seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/reader/OrcReadStrategy.java: ## @@ -

Re: [PR] [LDAP][Seatunnel-web] Add ldap support for seatunnel web api calls [seatunnel-web]

2025-01-14 Thread via GitHub
arshadmohammad commented on PR #245: URL: https://github.com/apache/seatunnel-web/pull/245#issuecomment-2590898698 Thank you, @BilwaST, for your confirmation. @shashwatsai, would you be able to raise a PR for this? -- This is an automated message from the Apache Git Service. To respond

Re: [I] [Feature][Docker] remove mysql lib from docker image [seatunnel]

2025-01-14 Thread via GitHub
Suvrat1629 commented on issue #8146: URL: https://github.com/apache/seatunnel/issues/8146#issuecomment-2591867596 @davidzollo I would like to understand the codebase and the repository a bit. Also for the issue which file do i need to look into? I am finding the codebase too big to navi

(seatunnel) branch dev updated: [Improve][Doc] Update the transform contribute guide (#8487)

2025-01-14 Thread wanghailin
This is an automated email from the ASF dual-hosted git repository. wanghailin pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new 875fcd89c5 [Improve][Doc] Update the transform

Re: [PR] [Improve][Doc] Update the transform contribute guide [seatunnel]

2025-01-14 Thread via GitHub
hailin0 merged PR #8487: URL: https://github.com/apache/seatunnel/pull/8487 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunne

Re: [PR] [Feature][Iceberg] Support read multi-table [seatunnel]

2025-01-14 Thread via GitHub
hailin0 commented on PR #8524: URL: https://github.com/apache/seatunnel/pull/8524#issuecomment-2591734344 https://github.com/user-attachments/assets/347561cf-598b-4d55-a001-6c21fe0e807b"; /> -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] [Bug] [mysql-cdc] when the startup mode is startup.mode="earliest", restoration cannot be performed [seatunnel]

2025-01-14 Thread via GitHub
jw-itq commented on issue #8522: URL: https://github.com/apache/seatunnel/issues/8522#issuecomment-2591735830 I replicated this issue locally, requiring mysql5.7 and enabling gtid https://github.com/user-attachments/assets/d9fcd25d-0c82-40d0-895f-7370fa56fb51"; /> -- This is an automat

Re: [PR] [Improve] [Connector-V2] support TTL for multiple table sink writer [seatunnel]

2025-01-14 Thread via GitHub
Hisoka-X commented on PR #8398: URL: https://github.com/apache/seatunnel/pull/8398#issuecomment-2591739020 Please resolve conflicts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[PR] [Fix][Doc] Fix dead link [seatunnel]

2025-01-14 Thread via GitHub
Hisoka-X opened a new pull request, #8525: URL: https://github.com/apache/seatunnel/pull/8525 ### Purpose of this pull request Fix dead link cause by #8487 ### Does this PR introduce _any_ user-facing change? ### How was this patch tested?

Re: [I] [Bug] [JDBC] ORA-01461 [seatunnel]

2025-01-14 Thread via GitHub
zichen0907 commented on issue #7418: URL: https://github.com/apache/seatunnel/issues/7418#issuecomment-2591704393 how to solve the problem? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] [hotfix] [Connector-v2][Elasticsearch] Fix the timestamp conversion issue [seatunnel]

2025-01-14 Thread via GitHub
Hisoka-X commented on code in PR #8481: URL: https://github.com/apache/seatunnel/pull/8481#discussion_r1915979656 ## seatunnel-connectors-v2/connector-elasticsearch/src/main/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/serialize/source/DefaultSeaTunnelRowDeserializer.jav

Re: [PR] [Doc][Translate] translated and corrected the error in the original document [seatunnel]

2025-01-14 Thread via GitHub
Hisoka-X commented on PR #8505: URL: https://github.com/apache/seatunnel/pull/8505#issuecomment-2591690543 Please fix ci @iamadamnBoy -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [Fix][connector-elasticsearch] support elasticsearch nest type && spark with Array [seatunnel]

2025-01-14 Thread via GitHub
Hisoka-X commented on code in PR #8492: URL: https://github.com/apache/seatunnel/pull/8492#discussion_r1915971645 ## seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/SeaTunnelRow.java: ## @@ -169,7 +169,9 @@ private int getBytesForValue(Object v, SeaTunnelDataType

Re: [PR] [Doc][Translate] translated and corrected the error in the original document [seatunnel]

2025-01-14 Thread via GitHub
iamadamnBoy commented on PR #8505: URL: https://github.com/apache/seatunnel/pull/8505#issuecomment-2591782289 fixed already -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] [Feature][Iceberg] Support read multi-table [seatunnel]

2025-01-14 Thread via GitHub
hailin0 opened a new pull request, #8524: URL: https://github.com/apache/seatunnel/pull/8524 ### Purpose of this pull request ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? ### Check list

Re: [I] [Discuss][seatunnel-api] How can a user-defined ConfigShade class load external files [seatunnel]

2025-01-14 Thread via GitHub
litiliu commented on issue #8494: URL: https://github.com/apache/seatunnel/issues/8494#issuecomment-2591546926 I think we can add kv pairs in the values env block, like this ` "env": { "jobMode": "batch", "parallelism": 1, "shade.identifier": "base64", "sh

Re: [I] [Discuss][seatunnel-api] How can a user-defined ConfigShade class load external files [seatunnel]

2025-01-14 Thread via GitHub
litiliu commented on issue #8494: URL: https://github.com/apache/seatunnel/issues/8494#issuecomment-2591554544 I'm glad to take this feature. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] The speed of data transfer decrease gradually [seatunnel]

2025-01-14 Thread via GitHub
Seeumt commented on issue #8521: URL: https://github.com/apache/seatunnel/issues/8521#issuecomment-2591396585 [seatunnel-engine-worker.log](https://github.com/user-attachments/files/18417559/seatunnel-engine-worker.log) This attachment is engine-worker-log -- This is an automated messa

Re: [PR] [Fix] [Kafka Source] kafka source use topic as table name instead of fullName [seatunnel]

2025-01-14 Thread via GitHub
Cheun99 commented on code in PR #8401: URL: https://github.com/apache/seatunnel/pull/8401#discussion_r1915845676 ## seatunnel-e2e/seatunnel-connector-v2-e2e/connector-kafka-e2e/src/test/java/org/apache/seatunnel/e2e/connector/kafka/KafkaIT.java: ## @@ -295,7 +315,7 @@ public voi

(seatunnel) branch dev updated: [Fix] [Kafka Source] kafka source use topic as table name instead of fullName (#8401)

2025-01-14 Thread fanjia
This is an automated email from the ASF dual-hosted git repository. fanjia pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new 3d4f4bb33a [Fix] [Kafka Source] kafka source use to

Re: [I] [Bug] [Kafka Source] Failed when the consume topic name like a.b.c.d [seatunnel]

2025-01-14 Thread via GitHub
Hisoka-X closed issue #8396: [Bug] [Kafka Source] Failed when the consume topic name like a.b.c.d URL: https://github.com/apache/seatunnel/issues/8396 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[I] [Bug] [mysql-cdc] when the startup mode is startup.mode="earliest", restoration cannot be performed [seatunnel]

2025-01-14 Thread via GitHub
jw-itq opened a new issue, #8522: URL: https://github.com/apache/seatunnel/issues/8522 ### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened

Re: [PR] [Fix] [Kafka Source] kafka source use topic as table name instead of fullName [seatunnel]

2025-01-14 Thread via GitHub
Hisoka-X merged PR #8401: URL: https://github.com/apache/seatunnel/pull/8401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunn

[PR] [core][doc][enhance]add sensitive columns and enhance the doc [seatunnel]

2025-01-14 Thread via GitHub
litiliu opened a new pull request, #8523: URL: https://github.com/apache/seatunnel/pull/8523 ### Purpose of this pull request This PR adds the aws access_key, secret_key to the DEFAULT_SENSITIVE_KEYWORDS, and optimizes the doc ### Does this PR introduce _any_ us

Re: [PR] [Fix][Doc] Fix dead link [seatunnel]

2025-01-14 Thread via GitHub
hailin0 merged PR #8525: URL: https://github.com/apache/seatunnel/pull/8525 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunne

Re: [I] [Bug] [Connector-V2] Bug postgres sink does not support composite primary key in jdbc mode [seatunnel]

2025-01-14 Thread via GitHub
shirukai commented on issue #8350: URL: https://github.com/apache/seatunnel/issues/8350#issuecomment-2591821832 I have the same issue, please assign it to me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

(seatunnel) branch dev updated: [Fix][Doc] Fix dead link (#8525)

2025-01-14 Thread wanghailin
This is an automated email from the ASF dual-hosted git repository. wanghailin pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new dbeeffbc82 [Fix][Doc] Fix dead link (#8525) dbe

Re: [I] [Bug] [Connector-V2] Bug postgres sink does not support composite primary key in jdbc mode [seatunnel]

2025-01-14 Thread via GitHub
liunaijie commented on issue #8350: URL: https://github.com/apache/seatunnel/issues/8350#issuecomment-2591828046 > I have the same issue, please assign it to me Looking forward for your contribute. -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [PR] [Bug] Disabled users are able to login [seatunnel-web]

2025-01-14 Thread via GitHub
arshadmohammad commented on PR #264: URL: https://github.com/apache/seatunnel-web/pull/264#issuecomment-2590914532 The functionality has been verified through integration testing. ![Screenshot from 2025-01-15 00-44-42](https://github.com/user-attachments/assets/e14475db-eeac-4b55-b382-642

Re: [I] [Feature][transform] support transform metrics [seatunnel]

2025-01-14 Thread via GitHub
github-actions[bot] closed issue #8172: [Feature][transform] support transform metrics URL: https://github.com/apache/seatunnel/issues/8172 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [I] [Bug] [Connecters] Seatunnel 混合集群模式,重启服务之后,connector 抛出 ClassCastException [seatunnel]

2025-01-14 Thread via GitHub
github-actions[bot] commented on issue #8180: URL: https://github.com/apache/seatunnel/issues/8180#issuecomment-2591383287 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is a

Re: [I] [Bug][Connector][JDBC] NVARCHAR2 type is not supported for DAMENG database table [seatunnel]

2025-01-14 Thread via GitHub
github-actions[bot] closed issue #8232: [Bug][Connector][JDBC] NVARCHAR2 type is not supported for DAMENG database table URL: https://github.com/apache/seatunnel/issues/8232 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] [Improve] [Connector-V2] [connector-elasticsearch]Sink Support inserting parent-child document data in versions prior to ES 6. x [seatunnel]

2025-01-14 Thread via GitHub
github-actions[bot] closed issue #8177: [Improve] [Connector-V2] [connector-elasticsearch]Sink Support inserting parent-child document data in versions prior to ES 6. x URL: https://github.com/apache/seatunnel/issues/8177 -- This is an automated message from the Apache Git Service. To respon

Re: [I] [Improve] [Connector-V2] [connector-elasticsearch]Sink Support inserting parent-child document data in versions prior to ES 6. x [seatunnel]

2025-01-14 Thread via GitHub
github-actions[bot] commented on issue #8177: URL: https://github.com/apache/seatunnel/issues/8177#issuecomment-2591383308 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is a

Re: [I] [Feature][transform] support transform metrics [seatunnel]

2025-01-14 Thread via GitHub
github-actions[bot] commented on issue #8172: URL: https://github.com/apache/seatunnel/issues/8172#issuecomment-2591383336 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is a

Re: [I] [Bug] [Connecters] Seatunnel 混合集群模式,重启服务之后,connector 抛出 ClassCastException [seatunnel]

2025-01-14 Thread via GitHub
github-actions[bot] closed issue #8180: [Bug] [Connecters] Seatunnel 混合集群模式,重启服务之后,connector 抛出 ClassCastException URL: https://github.com/apache/seatunnel/issues/8180 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] [Improve][Connector-file-base] In large file scenarios, split the single file into multiple shards [seatunnel]

2025-01-14 Thread via GitHub
Hisoka-X commented on code in PR #8507: URL: https://github.com/apache/seatunnel/pull/8507#discussion_r1915904441 ## seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/reader/OrcReadStrategy.java: ## @@

Re: [PR] [Fix][Connector-V2] Fixed adding table comments [seatunnel]

2025-01-14 Thread via GitHub
Hisoka-X commented on code in PR #8514: URL: https://github.com/apache/seatunnel/pull/8514#discussion_r1915909982 ## seatunnel-connectors-v2/connector-doris/src/main/java/org/apache/seatunnel/connectors/doris/config/DorisSinkOptions.java: ## @@ -134,6 +134,9 @@ public interface

Re: [PR] [Doc][Translate] translated and corrected the error in the original document [seatunnel]

2025-01-14 Thread via GitHub
Hisoka-X commented on code in PR #8505: URL: https://github.com/apache/seatunnel/pull/8505#discussion_r1915912361 ## docs/zh/connector-v2/source/Mysql.md: ## @@ -0,0 +1,314 @@ +# MySQL + +> JDBC Mysql 源连接器 + +## 描述 + +通过 JDBC 读取外部数据源数据。 + +## 支持 Mysql 版本 + +- 5.5/5.6/5.7/8.0/8.1

Re: [PR] [Fix][Connector-V2] Fixed adding table comments [seatunnel]

2025-01-14 Thread via GitHub
corgy-w commented on code in PR #8514: URL: https://github.com/apache/seatunnel/pull/8514#discussion_r1915911191 ## seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/config/ClickhouseConfig.java: ## @@ -228,7 +228,11

(seatunnel) branch dev updated: [Doc][Translate] translated and corrected the error in the original document (#8505)

2025-01-14 Thread wuchunfu
This is an automated email from the ASF dual-hosted git repository. wuchunfu pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new 2aaee5d731 [Doc][Translate] translated and corre

Re: [PR] [Doc][Translate] translated and corrected the error in the original document [seatunnel]

2025-01-14 Thread via GitHub
wuchunfu merged PR #8505: URL: https://github.com/apache/seatunnel/pull/8505 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunn

[PR] [LDAP][Seatunnel-web] Add LDAP Authentication Provider Support. [seatunnel-web]

2025-01-14 Thread via GitHub
shashwatsai opened a new pull request, #269: URL: https://github.com/apache/seatunnel-web/pull/269 ## Purpose of this pull request We need to add support for LDAP authentication. It is important to centralise user identification with already existing authentication service

Re: [I] Error reading clickhouse data [seatunnel]

2025-01-14 Thread via GitHub
zhilinli123 commented on issue #8512: URL: https://github.com/apache/seatunnel/issues/8512#issuecomment-2591834218 @YTQ419 Please provide the fishlog4.shard user event DDL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub