[I] [Bug] [Paimon] 动态分桶表写入后数据重复了 [seatunnel]

2025-01-20 Thread via GitHub
Pandas886 opened a new issue, #8565: URL: https://github.com/apache/seatunnel/issues/8565 ### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened

Re: [PR] [Feature][Core] Add slot allocation strategy [seatunnel]

2025-01-20 Thread via GitHub
Hisoka-X commented on code in PR #8233: URL: https://github.com/apache/seatunnel/pull/8233#discussion_r1922994789 ## docs/en/seatunnel-engine/slot-allocation-strategy.md: ## @@ -0,0 +1,117 @@ +# Slot Allocation Strategy + +Slot allocation strategy is an important part of SeaTunn

(seatunnel) branch dev updated (021af147cc -> 2bfb97e502)

2025-01-20 Thread wanghailin
This is an automated email from the ASF dual-hosted git repository. wanghailin pushed a change to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git from 021af147cc [Feature] [Postgre CDC]support array type (#8560) add 2bfb97e502 [Feature][Iceberg] Support read m

Re: [PR] [Feature][Iceberg] Support read multi-table [seatunnel]

2025-01-20 Thread via GitHub
hailin0 merged PR #8524: URL: https://github.com/apache/seatunnel/pull/8524 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunne

Re: [I] [Bug] [Engine] Frequent "TaskGroupLocation: TaskGroupLocation xxx already exists" error during task execution, causing task failure. [seatunnel]

2025-01-20 Thread via GitHub
kwonder0926 commented on issue #8566: URL: https://github.com/apache/seatunnel/issues/8566#issuecomment-2603751842 I saw a statement online that if a table to be transferred is too large, the checkpoint timeout needs to be set longer. But what value should I set it to? I would like to ask i

Re: [PR] [Feature][seatunnel-transforms-v2] LLM transform Support Ollama return format #8550 [seatunnel]

2025-01-20 Thread via GitHub
SplitfireUptown commented on code in PR #8551: URL: https://github.com/apache/seatunnel/pull/8551#discussion_r1923080697 ## seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/nlpmodel/llm/remote/custom/CustomModel.java: ## @@ -94,8 +95,15 @@ protected List chat

Re: [PR] [Feature][seatunnel-transforms-v2] LLM transform Support Ollama return format #8550 [seatunnel]

2025-01-20 Thread via GitHub
corgy-w commented on code in PR #8551: URL: https://github.com/apache/seatunnel/pull/8551#discussion_r1923086606 ## seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/nlpmodel/llm/remote/custom/CustomModel.java: ## @@ -94,8 +95,15 @@ protected List chatWithMode

Re: [I] [Bug] [MySQL-CDC to Doris] 流式任务状态显示失败但是任务仍在运行,也无法手工停止 [seatunnel]

2025-01-20 Thread via GitHub
kwonder0926 commented on issue #8065: URL: https://github.com/apache/seatunnel/issues/8065#issuecomment-2603524677 我也一直遇到这个问题 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] [Bug] [Sftp Reader] If there are wildcard file names in the sftp directory and there happen to be other matching file names, a uniqueness error will occur [seatunnel]

2025-01-20 Thread via GitHub
hailin0 closed issue #8500: [Bug] [Sftp Reader] If there are wildcard file names in the sftp directory and there happen to be other matching file names, a uniqueness error will occur URL: https://github.com/apache/seatunnel/issues/8500 -- This is an automated message from the Apache Git Serv

Re: [I] [Feature][Connector - V2] Support for AWS Glue Data Catalog in Apache Iceberg connector [seatunnel]

2025-01-20 Thread via GitHub
rafi12396 commented on issue #8559: URL: https://github.com/apache/seatunnel/issues/8559#issuecomment-2603813172 @litiliu , understood, thanks for clarifying, will try it out. It was a little unclear from the docs. -- This is an automated message from the Apache Git Service. To respond to

Re: [I] 【2.3.9】【又来BUG了】mysql到starrocks跑批,source加上过滤条件,结果读不到数据,导致写SR不生效 [seatunnel]

2025-01-20 Thread via GitHub
andyuan1000 commented on issue #8567: URL: https://github.com/apache/seatunnel/issues/8567#issuecomment-2603813170 11 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [I] seatunnel 2.3.8有没有可以通过restapi执行hocon格式文件的方式 [seatunnel]

2025-01-20 Thread via GitHub
fcb-xiaobo commented on issue #8563: URL: https://github.com/apache/seatunnel/issues/8563#issuecomment-2603461582 Version 2.3.9 has a method of submitting tasks by uploading files through the REST API. You can refer to it below https://seatunnel.apache.org/docs/2.3.9/seatunnel-engine/res

Re: [PR] [Feature][Core] Add slot allocation strategy [seatunnel]

2025-01-20 Thread via GitHub
Hisoka-X commented on code in PR #8233: URL: https://github.com/apache/seatunnel/pull/8233#discussion_r1922989798 ## docs/en/seatunnel-engine/slot-allocation-strategy.md: ## @@ -0,0 +1,117 @@ +# Slot Allocation Strategy Review Comment: Please update https://github.com/apach

[I] [Bug] [Engine] Frequent "TaskGroupLocation: TaskGroupLocation xxx already exists" error during task execution, causing task failure. [seatunnel]

2025-01-20 Thread via GitHub
kwonder0926 opened a new issue, #8566: URL: https://github.com/apache/seatunnel/issues/8566 ### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happen

[I] [Bug] [seatunnel-engine] close pending status job failed [seatunnel]

2025-01-20 Thread via GitHub
Xuzhengz opened a new issue, #8564: URL: https://github.com/apache/seatunnel/issues/8564 ### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened

Re: [I] seatunnel 2.3.8有没有可以通过restapi执行hocon格式文件的方式 [seatunnel]

2025-01-20 Thread via GitHub
soy-beanmilk commented on issue #8563: URL: https://github.com/apache/seatunnel/issues/8563#issuecomment-2603549430 最好的方式是restapi能够支持hocon格式,如果支持不了的话,能不能提供hacon格式转换为json格式的工具,或者文档说明,因为我发现官方文档对于json格式这块描述的不是很多。我尝试用一些工具直接将hacon格式转换为json,发现是行不通的,因为有一些字段差异。 -- This is an automated message

Re: [I] seatunnel 2.3.8有没有可以通过restapi执行hocon格式文件的方式 [seatunnel]

2025-01-20 Thread via GitHub
soy-beanmilk commented on issue #8563: URL: https://github.com/apache/seatunnel/issues/8563#issuecomment-2603481457 但是我们暂时还不能升级seatunnel版本,目前有没有hacon转json格式详细说明,或者相关工具。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [I] 【2.3.9】mysql到starrocks跑批,加上过滤条件source打印为空,不生效 [seatunnel]

2025-01-20 Thread via GitHub
andyuan1000 commented on issue #8567: URL: https://github.com/apache/seatunnel/issues/8567#issuecomment-2603783140 ![Image](https://github.com/user-attachments/assets/d637a8bc-0732-4af7-b02a-c3dfbfeda79d) -- This is an automated message from the Apache Git Service. To respond to the messa

[I] 【2.3.9】mysql到starrocks跑批,加上过滤条件source打印为空,不生效 [seatunnel]

2025-01-20 Thread via GitHub
andyuan1000 opened a new issue, #8567: URL: https://github.com/apache/seatunnel/issues/8567 ### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happen

Re: [I] 【2.3.9】【又来BUG了】mysql到starrocks跑批,source加上过滤条件,结果读不到数据,导致写SR不生效 [seatunnel]

2025-01-20 Thread via GitHub
andyuan1000 closed issue #8567: 【2.3.9】【又来BUG了】mysql到starrocks跑批,source加上过滤条件,结果读不到数据,导致写SR不生效 URL: https://github.com/apache/seatunnel/issues/8567 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] [Feature][seatunnel-transforms-v2] LLM transform Support Ollama return format #8550 [seatunnel]

2025-01-20 Thread via GitHub
corgy-w commented on code in PR #8551: URL: https://github.com/apache/seatunnel/pull/8551#discussion_r1923039668 ## seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/nlpmodel/llm/remote/custom/CustomModel.java: ## @@ -94,8 +95,15 @@ protected List chatWithMode

Re: [I] [Feature][Connector - V2] Support for AWS Glue Data Catalog in Apache Iceberg connector [seatunnel]

2025-01-20 Thread via GitHub
litiliu commented on issue #8559: URL: https://github.com/apache/seatunnel/issues/8559#issuecomment-2603639065 I think the glue catalog has already been supported, you just need to set the params in the iceberg.catalog.config, like [iceberg.catalog.config](https://iceberg.apache.org/docs/1.

(seatunnel) branch dev updated: [Fix][connector-elasticsearch] support elasticsearch nest type && spark with Array (#8492)

2025-01-20 Thread wanghailin
This is an automated email from the ASF dual-hosted git repository. wanghailin pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new 92d2a4a106 [Fix][connector-elasticsearch] suppo

(seatunnel) branch dev updated: [Hotfix][Connector-V2][SFTP] Add quote to sftp file names with wildcard characters (#8501)

2025-01-20 Thread wanghailin
This is an automated email from the ASF dual-hosted git repository. wanghailin pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new c5751b001b [Hotfix][Connector-V2][SFTP] Add quo

Re: [PR] [Hotfix][Connector-V2][SFTP] Add quote to sftp file names with wildcard characters [seatunnel]

2025-01-20 Thread via GitHub
hailin0 merged PR #8501: URL: https://github.com/apache/seatunnel/pull/8501 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunne

Re: [I] seatunnel 2.3.8有没有可以通过restapi执行hocon格式文件的方式 [seatunnel]

2025-01-20 Thread via GitHub
fcb-xiaobo commented on issue #8563: URL: https://github.com/apache/seatunnel/issues/8563#issuecomment-2603527438 In 2.3.8, do you want to submit a hocon format configuration file through the REST API to submit the task, or do you want to convert the hocon configuration to JSON configuratio

Re: [PR] [Fix][connector-elasticsearch] support elasticsearch nest type && spark with Array [seatunnel]

2025-01-20 Thread via GitHub
hailin0 merged PR #8492: URL: https://github.com/apache/seatunnel/pull/8492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunne

(seatunnel) branch dev updated: [Feature] [Postgre CDC]support array type (#8560)

2025-01-20 Thread wanghailin
This is an automated email from the ASF dual-hosted git repository. wanghailin pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new 021af147cc [Feature] [Postgre CDC]support array

Re: [I] [Feature] [Postgre CDC] Unsupported type: ARRAY [seatunnel]

2025-01-20 Thread via GitHub
hailin0 closed issue #8466: [Feature] [Postgre CDC] Unsupported type: ARRAY URL: https://github.com/apache/seatunnel/issues/8466 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [Feature] [Postgre CDC]support array type [seatunnel]

2025-01-20 Thread via GitHub
hailin0 merged PR #8560: URL: https://github.com/apache/seatunnel/pull/8560 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunne

Re: [PR] [Fix][connector-elasticsearch] support elasticsearch nest type && spark with Array [seatunnel]

2025-01-20 Thread via GitHub
Carl-Zhou-CN commented on code in PR #8492: URL: https://github.com/apache/seatunnel/pull/8492#discussion_r1923005770 ## seatunnel-translation/seatunnel-translation-spark/seatunnel-translation-spark-common/src/main/java/org/apache/seatunnel/translation/spark/serialization/InternalRowCon

Re: [I] [Feature][Connector-V2] Support GraphQL Connector [seatunnel]

2025-01-20 Thread via GitHub
eyys commented on issue #8557: URL: https://github.com/apache/seatunnel/issues/8557#issuecomment-2603531532 Hi, can I try to finish the development of this feature -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] [Bug] Task instance filter by execution user is not working [seatunnel-web]

2025-01-20 Thread via GitHub
arshadmohammad commented on PR #275: URL: https://github.com/apache/seatunnel-web/pull/275#issuecomment-2602251196 After fix, filter by execute user, is working fine. ![image](https://github.com/user-attachments/assets/cde72230-fdd0-4a6d-8a2d-04e4c52ba81e) -- This is an automated m

[I] [Bug] [Seatunnel-web] Incorrect Task Instance Filter Options Displayed [seatunnel]

2025-01-20 Thread via GitHub
arshadmohammad opened a new issue, #8561: URL: https://github.com/apache/seatunnel/issues/8561 ### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What hap

[PR] [Bug] Task instance filter by execution user is not working [seatunnel-web]

2025-01-20 Thread via GitHub
arshadmohammad opened a new pull request, #275: URL: https://github.com/apache/seatunnel-web/pull/275 ## Purpose of this pull request ## Check list * [ ] Code changed are covered with tests, or it does not need tests for reason: * [ ] If any new Jar binary pa

Re: [I] [Bug] [Seatunnel-web] Incorrect Task Instance Filter Options Displayed [seatunnel]

2025-01-20 Thread via GitHub
arshadmohammad commented on issue #8561: URL: https://github.com/apache/seatunnel/issues/8561#issuecomment-2602238530 If anyone is interested in working on this issue, please feel free to assign it to yourself. -- This is an automated message from the Apache Git Service. To respond to the

[I] [Bug] [MongoDB-CDC] Can not find catalog with factoryId [MongoDB] [seatunnel]

2025-01-20 Thread via GitHub
amol64546 opened a new issue, #8562: URL: https://github.com/apache/seatunnel/issues/8562 ### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened

Re: [I] [Bug][Sink][HdfsFile] Parameters: data_save_mode and schema_save_mode do not take effect [seatunnel]

2025-01-20 Thread via GitHub
github-actions[bot] commented on issue #8289: URL: https://github.com/apache/seatunnel/issues/8289#issuecomment-2603404593 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is a

Re: [I] [Bug][Sink][HdfsFile] Parameters: data_save_mode and schema_save_mode do not take effect [seatunnel]

2025-01-20 Thread via GitHub
github-actions[bot] closed issue #8289: [Bug][Sink][HdfsFile] Parameters: data_save_mode and schema_save_mode do not take effect URL: https://github.com/apache/seatunnel/issues/8289 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[PR] [Feature] [Postgre CDC]support array type [seatunnel]

2025-01-20 Thread via GitHub
litiliu opened a new pull request, #8560: URL: https://github.com/apache/seatunnel/pull/8560 ### Purpose of this pull request This close #8466 ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? UT

Re: [PR] [feature][core] Unified engine initialization connector logic [seatunnel]

2025-01-20 Thread via GitHub
liugddx commented on code in PR #8536: URL: https://github.com/apache/seatunnel/pull/8536#discussion_r1922045077 ## seatunnel-api/src/main/java/org/apache/seatunnel/api/table/factory/FactoryUtil.java: ## @@ -65,31 +73,53 @@ public final class FactoryUtil { public static

Re: [PR] [feature][core] Unified engine initialization connector logic [seatunnel]

2025-01-20 Thread via GitHub
liugddx commented on code in PR #8536: URL: https://github.com/apache/seatunnel/pull/8536#discussion_r1922045663 ## seatunnel-api/src/main/java/org/apache/seatunnel/api/table/factory/FactoryUtil.java: ## @@ -351,4 +401,29 @@ public static SeaTunnelTransform createAndPrepareMult

Re: [PR] [feature][core] Unified engine initialization connector logic [seatunnel]

2025-01-20 Thread via GitHub
liugddx commented on code in PR #8536: URL: https://github.com/apache/seatunnel/pull/8536#discussion_r1922045982 ## seatunnel-plugin-discovery/src/main/java/org/apache/seatunnel/plugin/discovery/AbstractPluginDiscovery.java: ## @@ -75,12 +76,24 @@ public abstract class AbstractP