[I] [Bug] [k8s_Deploy] The client submits tasks and the home directory of the cluster must be consistent?? [seatunnel]

2024-12-24 Thread via GitHub
reformdai opened a new issue, #8371: URL: https://github.com/apache/seatunnel/issues/8371 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened

Re: [PR] [Hotfix][Transform][Sql] function concat_ws handle array type, avoid write [Ljava.lang.String. [seatunnel]

2024-12-24 Thread via GitHub
Hisoka-X commented on code in PR #8369: URL: https://github.com/apache/seatunnel/pull/8369#discussion_r1896638589 ## seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/sql/zeta/ConcatWsFunctionTest.java: ## @@ -0,0 +1,47 @@ +package org.apache.seatunnel.transfo

Re: [PR] [Feature][K8S deploy] add helm chart [seatunnel]

2024-12-24 Thread via GitHub
hailin0 merged PR #8291: URL: https://github.com/apache/seatunnel/pull/8291 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunne

(seatunnel) branch dev updated: [Feature][K8S deploy] add helm chart (#8291)

2024-12-24 Thread wanghailin
This is an automated email from the ASF dual-hosted git repository. wanghailin pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new a1c90015d8 [Feature][K8S deploy] add helm chart

[PR] Bump ch.qos.logback:logback-core from 1.2.3 to 1.5.13 [seatunnel]

2024-12-24 Thread via GitHub
dependabot[bot] opened a new pull request, #8372: URL: https://github.com/apache/seatunnel/pull/8372 Bumps [ch.qos.logback:logback-core](https://github.com/qos-ch/logback) from 1.2.3 to 1.5.13. Commits https://github.com/qos-ch/logback/commit/32638aa7e99c0135cb1b81806ed05352e6b

(seatunnel) branch dependabot/maven/ch.qos.logback-logback-core-1.5.13 created (now b3c46326f3)

2024-12-24 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/ch.qos.logback-logback-core-1.5.13 in repository https://gitbox.apache.org/repos/asf/seatunnel.git at b3c46326f3 Bump ch.qos.logback:logback-core from 1.2.3 to 1.5.13

(seatunnel) branch dev updated: [Core][Flink] Resolve streaming source high CPU usage (#8354)

2024-12-24 Thread tyrantlucifer
This is an automated email from the ASF dual-hosted git repository. tyrantlucifer pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new eae32bfb84 [Core][Flink] Resolve streaming s

Re: [PR] [Fix][Flink] Resolve streaming source high CPU usage [seatunnel]

2024-12-24 Thread via GitHub
TyrantLucifer merged PR #8354: URL: https://github.com/apache/seatunnel/pull/8354 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@se

Re: [I] [Bug] [Flink] 使用flink引擎执行Seatunnel cdc任务时,source端一直处于busy状态,这个是极不正常的 [seatunnel]

2024-12-24 Thread via GitHub
TyrantLucifer closed issue #8213: [Bug] [Flink] 使用flink引擎执行Seatunnel cdc任务时,source端一直处于busy状态,这个是极不正常的 URL: https://github.com/apache/seatunnel/issues/8213 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] [Feature][Connector-V2] Grok config is supported for file reading [seatunnel]

2024-12-24 Thread via GitHub
corgy-w opened a new pull request, #8376: URL: https://github.com/apache/seatunnel/pull/8376 ### Purpose of this pull request #8103 Currently, only the grok parameter can be configured for a read ### Does this PR introduce _any_ user-facing change? ### How was this pa

[PR] [improve][kafka] use config to set kafka deserialize error ignore parameter [seatunnel]

2024-12-24 Thread via GitHub
liunaijie opened a new pull request, #8377: URL: https://github.com/apache/seatunnel/pull/8377 ### Purpose of this pull request now, some desrialize use hardcode value to set the `deserialize error ignore` value. actually we have this config, use config to set the value

Re: [I] [Bug] [Connector-V2] Source Oracle-CDC Sink Clickhouse checkpoint timeout [seatunnel]

2024-12-24 Thread via GitHub
bwcxyk commented on issue #8364: URL: https://github.com/apache/seatunnel/issues/8364#issuecomment-2560854397 大概捋了一下 `seatunnel.sh -c oracle_to_clickhouse.conf`不加`--async` Ctrl+C 终止后再运行`seatunnel.sh -c oracle_to_clickhouse.conf`, 就会出现 checkpoint timeout 的问题。在 Clickhouse `truncate table

Re: [I] mysql-CDC [seatunnel]

2024-12-24 Thread via GitHub
huchao-hu commented on issue #6566: URL: https://github.com/apache/seatunnel/issues/6566#issuecomment-2560854723 @hailin0 It has been set up for 7 days but this problem still occurs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [I] mysql-CDC [seatunnel]

2024-12-24 Thread via GitHub
huchao-hu commented on issue #6566: URL: https://github.com/apache/seatunnel/issues/6566#issuecomment-2560855542 @Diamonddest Have you solved the problem so far? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] [Feature][Transform] Support merge shading-tables to one table [seatunnel]

2024-12-24 Thread via GitHub
hailin0 commented on code in PR #8360: URL: https://github.com/apache/seatunnel/pull/8360#discussion_r1896814695 ## seatunnel-translation/seatunnel-translation-spark/seatunnel-translation-spark-common/src/main/java/org/apache/seatunnel/translation/spark/serialization/SeaTunnelRowConvert

Re: [PR] [Feature][Connector-V2] Grok config is supported for file reading [seatunnel]

2024-12-24 Thread via GitHub
wuchunfu commented on PR #8376: URL: https://github.com/apache/seatunnel/pull/8376#issuecomment-2561214911 @corgy-w Can you add some commonly used built-in grok expressions? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] [Feature][Jdbc] Support sink ddl for postgresql [seatunnel]

2024-12-24 Thread via GitHub
hailin0 closed issue #8113: [Feature][Jdbc] Support sink ddl for postgresql URL: https://github.com/apache/seatunnel/issues/8113 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [Feature][Connector-V2] Grok config is supported for file reading [seatunnel]

2024-12-24 Thread via GitHub
corgy-w commented on PR #8376: URL: https://github.com/apache/seatunnel/pull/8376#issuecomment-2561227178 > @corgy-w Can you add some commonly used built-in grok expressions? Yes, I will complete these together with the tests later. @wuchunfu -- This is an automated message from th

Re: [PR] [Improve] Remove deprecated code for MultipleRowTyp [seatunnel]

2024-12-24 Thread via GitHub
hailin0 commented on code in PR #8272: URL: https://github.com/apache/seatunnel/pull/8272#discussion_r1896825041 ## seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/task/SeaTunnelSourceCollector.java: ## @@ -97,15 +94,6 @@ public void col

Re: [I] [Bug] [Module Name] HADOOP_HOME and hadoop.home.dir are unset [seatunnel]

2024-12-24 Thread via GitHub
fcb-xiaobo commented on issue #8365: URL: https://github.com/apache/seatunnel/issues/8365#issuecomment-2560933505 > Isn't there a local mode to boot? Do you still need to install hadoop? @hawk9821 This shouldn't affect usage, right? The task can still run normally --

Re: [I] [Bug] [Connector-V2] Source Oracle-CDC Sink Clickhouse checkpoint timeout [seatunnel]

2024-12-24 Thread via GitHub
bwcxyk commented on issue #8364: URL: https://github.com/apache/seatunnel/issues/8364#issuecomment-2560823253 After restart Clickhouse, the first submission was fine, and the checkpoint timeout was submitted again. -- This is an automated message from the Apache Git Service. To respond to

[PR] [Improve][Flink] Optimization of Kafka Source Busy Problem [seatunnel]

2024-12-24 Thread via GitHub
zhangshenghang opened a new pull request, #8374: URL: https://github.com/apache/seatunnel/pull/8374 ### Purpose of this pull request - link #8358 subtask : Kafka optimization ### Does this PR introduce _any_ user-facing change? ### How was this pat

[I] [Bug][Seatunnel-web] Job Update Fails When tasks are added or removed as part of the update [seatunnel]

2024-12-24 Thread via GitHub
arshadmohammad opened a new issue, #8373: URL: https://github.com/apache/seatunnel/issues/8373 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What hap

Re: [PR] [Hotfix][Transform][Sql] function concat_ws handle array type, avoid write [Ljava.lang.String. [seatunnel]

2024-12-24 Thread via GitHub
sohurdc commented on code in PR #8369: URL: https://github.com/apache/seatunnel/pull/8369#discussion_r1896687769 ## seatunnel-transforms-v2/src/test/java/org/apache/seatunnel/transform/sql/zeta/ConcatWsFunctionTest.java: ## @@ -0,0 +1,47 @@ +package org.apache.seatunnel.transfor

[PR] [Improve][Doc] Update connector v2 contribute guide [seatunnel]

2024-12-24 Thread via GitHub
Hisoka-X opened a new pull request, #8375: URL: https://github.com/apache/seatunnel/pull/8375 ### Purpose of this pull request Update connector v2 contribute guide, remove some expired words. ### Does this PR introduce _any_ user-facing change? no

Re: [PR] [Improve][Doc] Update connector v2 contribute guide [seatunnel]

2024-12-24 Thread via GitHub
hailin0 commented on code in PR #8375: URL: https://github.com/apache/seatunnel/pull/8375#discussion_r1896736716 ## seatunnel-connectors-v2/README.md: ## @@ -16,216 +16,248 @@ development at the current stage, and reduces the difficulty of merging. - ../`seatunnel-translation`

Re: [PR] [Feature][Core] Add slot allocation strategy [seatunnel]

2024-12-24 Thread via GitHub
zhangshenghang commented on PR #8233: URL: https://github.com/apache/seatunnel/pull/8233#issuecomment-2561089420 > Overall LGTM. Thanks @zhangshenghang . But I think we should merge this after 2.3.9 released. Ok, thank you for your review -- This is an automated message from the Ap

Re: [I] [Bug][Connector-V2][Assert] AssertSinkWriter supports multiple concurrent operations [seatunnel]

2024-12-24 Thread via GitHub
corgy-w commented on issue #8357: URL: https://github.com/apache/seatunnel/issues/8357#issuecomment-2561179337 I'm in 'rehabilitation training' [/dog], if I have time I can try this issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] [Feature][K8S deploy] add helm chart [seatunnel]

2024-12-24 Thread via GitHub
FrommyMind commented on PR #8291: URL: https://github.com/apache/seatunnel/pull/8291#issuecomment-2560953463 CI https://github.com/FrommyMind/seatunnel/actions/runs/12476135067 rerun succeed。 -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] [Bug][Seatunnel-web] Job update API fails when tasks are added or removed [seatunnel-web]

2024-12-24 Thread via GitHub
arshadmohammad commented on PR #249: URL: https://github.com/apache/seatunnel-web/pull/249#issuecomment-2561367528 Added two new test cases to verify the scenario and executed the related test cases, all are passing ![Screenshot from 2024-12-25 01-08-38](https://github.com/user-attachmen

Re: [PR] [Fix][Connector-V2] Add Filter for Partitions to Prevent Blocking in KafkaConsumer [seatunnel]

2024-12-24 Thread via GitHub
Carl-Zhou-CN commented on code in PR #8314: URL: https://github.com/apache/seatunnel/pull/8314#discussion_r1897026368 ## seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/source/KafkaSourceSplitEnumerator.java: ## @@ -307,10 +3

Re: [PR] [Fix][Doc] fix doc tip style and add document format specification [seatunnel]

2024-12-24 Thread via GitHub
bwcxyk commented on PR #8378: URL: https://github.com/apache/seatunnel/pull/8378#issuecomment-2561605390 > Thanks @bwcxyk ! Could you add a test case to check all docs in > > https://github.com/apache/seatunnel/blob/bd89d84fe655cd1b262e0cb1b83cbddc0d1a2f8d/seatunnel-ci-tools/src/test/

Re: [PR] [Fix][Doc] fix doc tip style and add document format specification [seatunnel]

2024-12-24 Thread via GitHub
bwcxyk commented on PR #8378: URL: https://github.com/apache/seatunnel/pull/8378#issuecomment-2561607659 Use [admonitions](https://docusaurus.io/docs/next/markdown-features/admonitions) to highlight important information. -- This is an automated message from the Apache Git Service. To re

Re: [PR] [Fix][Doc] fix doc tip style and add document format specification [seatunnel]

2024-12-24 Thread via GitHub
bwcxyk commented on PR #8378: URL: https://github.com/apache/seatunnel/pull/8378#issuecomment-2561607453 Use [admonitions](https://docusaurus.io/docs/next/markdown-features/admonitions) to highlight important information. -- This is an automated message from the Apache Git Service. To re

Re: [PR] [Feature][API] Support timestamp with timezone offset [seatunnel]

2024-12-24 Thread via GitHub
hailin0 commented on code in PR #8367: URL: https://github.com/apache/seatunnel/pull/8367#discussion_r1897111584 ## seatunnel-api/src/main/java/org/apache/seatunnel/api/table/converter/BasicDataConverter.java: ## @@ -74,6 +74,8 @@ default Object convert(SeaTunnelDataType typeDef

Re: [PR] [Feature][MySQL-CDC] Support database/table wildcards scan read [seatunnel]

2024-12-24 Thread via GitHub
Hisoka-X commented on code in PR #8323: URL: https://github.com/apache/seatunnel/pull/8323#discussion_r1897058560 ## seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql/src/main/java/org/apache/seatunnel/connectors/seatunnel/cdc/mysql/source/MySqlIncrementalSourceFactory.java: ###

[PR] [Feature][Jdbc] Support sink ddl for dameng [seatunnel]

2024-12-24 Thread via GitHub
hawk9821 opened a new pull request, #8380: URL: https://github.com/apache/seatunnel/pull/8380 ### Purpose of this pull request #8127 ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? DmSchemaChangeIT

Re: [I] [Bug] [Connector-V2] connector-maxcompute: The source reader may cause data duplication [seatunnel]

2024-12-24 Thread via GitHub
liangcw commented on issue #8379: URL: https://github.com/apache/seatunnel/issues/8379#issuecomment-2561670595 Failed to connect to github.com port 443 after 21089 ms: Couldn't connect to server. this code may fix it /* * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] [Improve] [Connector-V2] [connector-elasticsearch]Sink Support inserting parent-child document data in versions prior to ES 6. x [seatunnel]

2024-12-24 Thread via GitHub
magic-fh commented on PR #8231: URL: https://github.com/apache/seatunnel/pull/8231#issuecomment-2561684809 @Hisoka-X @hailin0 Test cases have been added, please help review the code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[PR] [Bug][Seatunnel-web] Job update API fails when tasks are added or removed [seatunnel-web]

2024-12-24 Thread via GitHub
arshadmohammad opened a new pull request, #249: URL: https://github.com/apache/seatunnel-web/pull/249 ## Purpose of this pull request ## Check list * [ ] Code changed are covered with tests, or it does not need tests for reason: * [ ] If any new Jar binary pa

[I] [Bug] [Connector-V2] connector-maxcompute: The source reader may cause data duplication [seatunnel]

2024-12-24 Thread via GitHub
liangcw opened a new issue, #8379: URL: https://github.com/apache/seatunnel/issues/8379 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happen

[PR] [Fix][Doc] fix doc tip style and add document format specification [seatunnel]

2024-12-24 Thread via GitHub
bwcxyk opened a new pull request, #8378: URL: https://github.com/apache/seatunnel/pull/8378 - Added a new note section in the document format specification, including tips, notes, and cautions. - Update the use of comments in Clickhouse, Hive, and LocalFile connector documentation. -

Re: [I] [Feature][Hudi] Support sink ddl [seatunnel]

2024-12-24 Thread via GitHub
github-actions[bot] commented on issue #8118: URL: https://github.com/apache/seatunnel/issues/8118#issuecomment-2561507652 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

Re: [I] [Feature][Kudu] Support sink ddl [seatunnel]

2024-12-24 Thread via GitHub
github-actions[bot] commented on issue #8119: URL: https://github.com/apache/seatunnel/issues/8119#issuecomment-2561507644 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

Re: [I] [Feature][Hive] Support sink ddl [seatunnel]

2024-12-24 Thread via GitHub
github-actions[bot] commented on issue #8117: URL: https://github.com/apache/seatunnel/issues/8117#issuecomment-2561507660 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

Re: [I] [Feature][Elasticsearch] Support sink ddl [seatunnel]

2024-12-24 Thread via GitHub
github-actions[bot] commented on issue #8116: URL: https://github.com/apache/seatunnel/issues/8116#issuecomment-2561507674 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

Re: [PR] [Feature][Transform] Support merge shading-tables to one table [seatunnel]

2024-12-24 Thread via GitHub
Carl-Zhou-CN commented on code in PR #8360: URL: https://github.com/apache/seatunnel/pull/8360#discussion_r1897019000 ## seatunnel-translation/seatunnel-translation-spark/seatunnel-translation-spark-common/src/main/java/org/apache/seatunnel/translation/spark/serialization/SeaTunnelRowCo

Re: [I] [Bug] [Module Name] HADOOP_HOME and hadoop.home.dir are unset [seatunnel]

2024-12-24 Thread via GitHub
huchao-hu commented on issue #8365: URL: https://github.com/apache/seatunnel/issues/8365#issuecomment-2561519769 @fcb-xiaobo Currently, the startup fails and cannot be used normally -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

(seatunnel) branch dev updated (eae32bfb84 -> bd89d84fe6)

2024-12-24 Thread jarvis
This is an automated email from the ASF dual-hosted git repository. jarvis pushed a change to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git from eae32bfb84 [Core][Flink] Resolve streaming source high CPU usage (#8354) add bd89d84fe6 [improve][API] support m

Re: [PR] [improve][API] support metrics interface in rest v2 [seatunnel]

2024-12-24 Thread via GitHub
liunaijie merged PR #8312: URL: https://github.com/apache/seatunnel/pull/8312 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatun

Re: [PR] [Feature][MySQL-CDC] Support database/table wildcards scan read [seatunnel]

2024-12-24 Thread via GitHub
Carl-Zhou-CN commented on code in PR #8323: URL: https://github.com/apache/seatunnel/pull/8323#discussion_r1897022823 ## seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql/src/main/java/org/apache/seatunnel/connectors/seatunnel/cdc/mysql/source/MySqlIncrementalSourceFactory.java:

Re: [PR] [improve][kafka] use config to set kafka deserialize error ignore parameter [seatunnel]

2024-12-24 Thread via GitHub
Carl-Zhou-CN commented on code in PR #8377: URL: https://github.com/apache/seatunnel/pull/8377#discussion_r1897023607 ## seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/source/KafkaSourceConfig.java: ## @@ -255,10 +255,13 @@

Re: [PR] [Feature][API] Support timestamp with timezone offset [seatunnel]

2024-12-24 Thread via GitHub
Carl-Zhou-CN commented on code in PR #8367: URL: https://github.com/apache/seatunnel/pull/8367#discussion_r1897024025 ## seatunnel-api/src/main/java/org/apache/seatunnel/api/table/converter/BasicDataConverter.java: ## @@ -74,6 +74,8 @@ default Object convert(SeaTunnelDataType ty