Re: [I] [Bug] [connector-file-s3] S3 sink data loss [seatunnel]

2025-04-09 Thread via GitHub
FrommyMind commented on issue #9133: URL: https://github.com/apache/seatunnel/issues/9133#issuecomment-2791375347 Do you mean you have a kafka to s3 job at the same time? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[I] [Feature][Module Name] tidb-cdc 现在只能单表同步 无法支持多表或者整库 [seatunnel]

2025-04-09 Thread via GitHub
lvbo1 opened a new issue, #9134: URL: https://github.com/apache/seatunnel/issues/9134 ### Search before asking - [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement. ###

[PR] [Feature][Transform] support boolean type for sql transform [seatunnel]

2025-04-09 Thread via GitHub
hawk9821 opened a new pull request, #9136: URL: https://github.com/apache/seatunnel/pull/9136 ### Purpose of this pull request support boolean type for sql transform. eg: select ( case when `col1` = 'a' then true else false end ) as `bool_1` , cast( `col2` as boolean) as `bo

Re: [I] [Bug] [connector-file-s3] S3 sink data loss [seatunnel]

2025-04-09 Thread via GitHub
cw-longwang commented on issue #9133: URL: https://github.com/apache/seatunnel/issues/9133#issuecomment-2791497690 ![Image](https://github.com/user-attachments/assets/6c2bb05e-e245-4473-a46b-3d85b11d658b) For stream processing tasks, there is always only one file for a task. Because the fi

Re: [I] [Bug] [connector-file-s3] S3 sink data loss [seatunnel]

2025-04-09 Thread via GitHub
cw-longwang commented on issue #9133: URL: https://github.com/apache/seatunnel/issues/9133#issuecomment-2791510090 [taskmanager.log](https://github.com/user-attachments/files/19677861/taskmanager.log) -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] [Bug] [connector-file-s3] S3 sink data loss [seatunnel]

2025-04-09 Thread via GitHub
cw-longwang commented on issue #9133: URL: https://github.com/apache/seatunnel/issues/9133#issuecomment-2791509492 ![Image](https://github.com/user-attachments/assets/4f982453-4483-4730-8713-49d259ef5522)Each parallelism, the file name is exactly the same each time -- This is an automate

Re: [I] [Bug] [connector-file-s3] S3 sink data loss [seatunnel]

2025-04-09 Thread via GitHub
cw-longwang commented on issue #9133: URL: https://github.com/apache/seatunnel/issues/9133#issuecomment-2791510476 @FrommyMind -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] [Feature][Transform] Support single/batch mode vectorization using Amazon Titan & cohere embedding model [seatunnel]

2025-04-09 Thread via GitHub
SEZ9 commented on PR #9120: URL: https://github.com/apache/seatunnel/pull/9120#issuecomment-2791606190 Hi @hailin0 @corgy-w . Is there anything else I need to do before this PR can be merged? -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] [Feature][Connector-V2] Support GraphQL Connector (#8557) [seatunnel]

2025-04-09 Thread via GitHub
eyys commented on PR #9021: URL: https://github.com/apache/seatunnel/pull/9021#issuecomment-2791354269 @hailin0 check done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] [Feature][Connector-V2] Support GraphQL Connector (#8557) [seatunnel]

2025-04-09 Thread via GitHub
eyys commented on PR #9021: URL: https://github.com/apache/seatunnel/pull/9021#issuecomment-2791358664 @hailin0 @Hisoka-X The test results are correct, but there seems to be a problem indicated here. ![Uploading Snipaste_2025-04-10_09-56-39.jpg…]() -- This is an automated message f

[I] [Bug] 在转换操作中文字段的时候,任务报错 [seatunnel]

2025-04-09 Thread via GitHub
tangyungteng opened a new issue, #9135: URL: https://github.com/apache/seatunnel/issues/9135 ### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happe

Re: [PR] [Feature] Add Workspace Support [seatunnel-web]

2025-04-09 Thread via GitHub
arshadmohammad commented on PR #284: URL: https://github.com/apache/seatunnel-web/pull/284#issuecomment-2791540130 @Hisoka-X can you please review and merge this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] [Feat][Connector-v2][Iceberg]support filter conditions in iceberg source [seatunnel]

2025-04-09 Thread via GitHub
litiliu commented on code in PR #9095: URL: https://github.com/apache/seatunnel/pull/9095#discussion_r2036383948 ## docs/en/connector-v2/source/Iceberg.md: ## @@ -112,6 +114,7 @@ source { } namespace = "database1" table = "source" +filter = "select * from tabl

[PR] [Feature] Add APIs to get resource names. [seatunnel-web]

2025-04-09 Thread via GitHub
arshadmohammad opened a new pull request, #286: URL: https://github.com/apache/seatunnel-web/pull/286 The resource name provider APIs will be used by Ranger seatunnel plugin to autocomplete resource names. ## Purpose of this pull request ## Check list * [ ] C

Re: [I] [Bug] [Transforms SQL] Unsupported function: SYSDATE [seatunnel]

2025-04-09 Thread via GitHub
github-actions[bot] commented on issue #8946: URL: https://github.com/apache/seatunnel/issues/8946#issuecomment-2791260885 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

Re: [I] [Feature][Connector] Add Apache Cloudberry Support [seatunnel]

2025-04-09 Thread via GitHub
github-actions[bot] commented on issue #8932: URL: https://github.com/apache/seatunnel/issues/8932#issuecomment-2791260921 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

Re: [I] kafka to S3 set "have_partition:true" has a problem,why? [seatunnel]

2025-04-09 Thread via GitHub
cw-longwang commented on issue #8909: URL: https://github.com/apache/seatunnel/issues/8909#issuecomment-2789736935 I have a problem writing to s3. The flink engine reads kafka and Mysqlcdc writes to s3. The two checkpoint file names are the same. As a result, the later file overwrites the

Re: [I] [Bug] [Hive-Source-Connector] zeta读取hive以数字开头的字段,报错 [seatunnel]

2025-04-09 Thread via GitHub
shashwatsai commented on issue #9113: URL: https://github.com/apache/seatunnel/issues/9113#issuecomment-2789745040 In Hive, column names starting with digits are not valid identifiers, quoting with backticks might work, as hive follows Hive follows SQL identifier rules loosely. You can try

[PR] [Feature] Add Support for Authorization [seatunnel-web]

2025-04-09 Thread via GitHub
arshadmohammad opened a new pull request, #285: URL: https://github.com/apache/seatunnel-web/pull/285 Authorization has been added for the Workspace, User, Datasource, and Job resources for the operations Create, Update, Delete, Read, and Execute. Seatunnel web has end User resources

Re: [I] [Bug] [Transform] Json parsing exception, value 'DEEPSEEK' [seatunnel]

2025-04-09 Thread via GitHub
BigTailMonkey commented on issue #8901: URL: https://github.com/apache/seatunnel/issues/8901#issuecomment-2791309439 I get the same bug. Caused by: java.lang.IllegalArgumentException: Json parsing exception, value 'DEEPSEEK', and expected type 'org.apache.seatunnel.transform.nlpmodel

[I] [Bug] [connector-file-s3] Bug title [seatunnel]

2025-04-09 Thread via GitHub
cw-longwang opened a new issue, #9133: URL: https://github.com/apache/seatunnel/issues/9133 ### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happen

Re: [I] [Bug] [connector-file-s3] S3 sink data loss [seatunnel]

2025-04-09 Thread via GitHub
cw-longwang commented on issue #9133: URL: https://github.com/apache/seatunnel/issues/9133#issuecomment-2791496017 @FrommyMind No. There is only one seatunnel task. In the same task, two checkpoints write two files, and the file name is exactly the same. -- This is an automated message fr

Re: [PR] [Feature][Transform] support boolean type for sql transform [seatunnel]

2025-04-09 Thread via GitHub
Copilot commented on code in PR #9136: URL: https://github.com/apache/seatunnel/pull/9136#discussion_r2036610449 ## seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/functions/SystemFunction.java: ## @@ -177,6 +178,26 @@ public static Object castAs(Li

[PR] [Fix][Zeta] Occasional checkpoint failed when use COS as checkpoint storage [seatunnel]

2025-04-09 Thread via GitHub
lm-ylj opened a new pull request, #9137: URL: https://github.com/apache/seatunnel/pull/9137 ### Purpose of this pull request close https://github.com/apache/seatunnel/issues/9124 ### Does this PR introduce _any_ user-facing change? ### How was this

Re: [PR] [Feature][Connector-V2] Support GraphQL Connector (#8557) [seatunnel]

2025-04-09 Thread via GitHub
eyys commented on PR #9021: URL: https://github.com/apache/seatunnel/pull/9021#issuecomment-2791359171 ![Snipaste_2025-04-10_09-56-39](https://github.com/user-attachments/assets/f2aa6fb5-ef1a-4d4d-b86f-e2d006e41009) -- This is an automated message from the Apache Git Service. To respo

Re: [I] [Bug] [Module Name] com.huawei.gauss.jdbc.ZenithDriver-1.2.1 [No suitable driver found] [seatunnel]

2025-04-09 Thread via GitHub
FrommyMind commented on issue #9132: URL: https://github.com/apache/seatunnel/issues/9132#issuecomment-2791377610 I think you should download the jdbc driver jar, and put into the lib path. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] [Feature] Add Workspace Support [seatunnel-web]

2025-04-09 Thread via GitHub
arshadmohammad commented on PR #284: URL: https://github.com/apache/seatunnel-web/pull/284#issuecomment-2789567990 JobExecutorControllerTest.storeErrorMessageWhenJobFailed is known flaky, rest all integration tests are passed. -- This is an automated message from the Apache Git Service. T

[I] [Feature][Seatunnel-We][Workspace] Add select workspace option in login page [seatunnel]

2025-04-09 Thread via GitHub
arshadmohammad opened a new issue, #9131: URL: https://github.com/apache/seatunnel/issues/9131 ### Search before asking - [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.

Re: [I] [Bug] [connectors-v2] 使用localfile 本地导入超过50万行的excel,速度越来越慢,提示oom [seatunnel]

2025-04-09 Thread via GitHub
dwave commented on issue #8040: URL: https://github.com/apache/seatunnel/issues/8040#issuecomment-2789782737 > How to reproduce this issue Test with an Excel file of 1,000,000 records on a single sheet page -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] [Feature] Add Workspace Support [seatunnel-web]

2025-04-09 Thread via GitHub
arshadmohammad commented on PR #284: URL: https://github.com/apache/seatunnel-web/pull/284#issuecomment-2789495978 After the change, jobs are executig successfully ![Screenshot from 2025-04-09 17-43-05](https://github.com/user-attachments/assets/17e70d49-72bc-4d0b-bdb3-232435fa)

Re: [I] kafka to S3 set "have_partition:true" has a problem,why? [seatunnel]

2025-04-09 Thread via GitHub
cw-longwang commented on issue #8909: URL: https://github.com/apache/seatunnel/issues/8909#issuecomment-2789738247 conf # Defining the runtime environment env { parallelism = 1 job.mode = "STREAMING" checkpoint.interval=6 } source { MySQL-CDC {

Re: [PR] [Feature] Add Workspace Support [seatunnel-web]

2025-04-09 Thread via GitHub
arshadmohammad commented on PR #284: URL: https://github.com/apache/seatunnel-web/pull/284#issuecomment-2789571378 Hi All, my testing is done, please review and merge this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] [Feat][Connector-v2][Iceberg]support filter conditions in iceberg source [seatunnel]

2025-04-09 Thread via GitHub
hailin0 commented on code in PR #9095: URL: https://github.com/apache/seatunnel/pull/9095#discussion_r2035527573 ## docs/en/connector-v2/source/Iceberg.md: ## @@ -112,6 +114,7 @@ source { } namespace = "database1" table = "source" +filter = "select * from tabl

Re: [PR] [Feature] Add Workspace Support [seatunnel-web]

2025-04-09 Thread via GitHub
arshadmohammad commented on PR #284: URL: https://github.com/apache/seatunnel-web/pull/284#issuecomment-2789565768 ![Screenshot from 2025-04-09 18-09-10](https://github.com/user-attachments/assets/a3ec31fe-553d-4fb9-84c2-1d3190a7bd06) -- This is an automated message from the Apache Gi

Re: [I] [Bug] [Jdbc] how to use DataSaveMode [seatunnel]

2025-04-09 Thread via GitHub
misi1987107 commented on issue #5366: URL: https://github.com/apache/seatunnel/issues/5366#issuecomment-2788617108 Issue Cannot Reproduce -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

(seatunnel) branch dev updated: [Fix][Connector-V2] Fix postgres cdc with debezium_json format can not parse number without scale (#9052)

2025-04-09 Thread fanjia
This is an automated email from the ASF dual-hosted git repository. fanjia pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new 29cf3a76c7 [Fix][Connector-V2] Fix postgres cdc wit

[PR] Bump vite from 5.4.9 to 5.4.17 in /seatunnel-engine/seatunnel-engine-ui [seatunnel]

2025-04-09 Thread via GitHub
dependabot[bot] opened a new pull request, #9130: URL: https://github.com/apache/seatunnel/pull/9130 Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 5.4.9 to 5.4.17. Release notes Sourced from https://github.com/vitejs/vite/releases";>vite's releases.

Re: [PR] Bump vite from 5.4.9 to 5.4.16 in /seatunnel-engine/seatunnel-engine-ui [seatunnel]

2025-04-09 Thread via GitHub
dependabot[bot] commented on PR #9094: URL: https://github.com/apache/seatunnel/pull/9094#issuecomment-2788952579 Superseded by #9130. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Bump vite from 5.4.9 to 5.4.16 in /seatunnel-engine/seatunnel-engine-ui [seatunnel]

2025-04-09 Thread via GitHub
dependabot[bot] closed pull request #9094: Bump vite from 5.4.9 to 5.4.16 in /seatunnel-engine/seatunnel-engine-ui URL: https://github.com/apache/seatunnel/pull/9094 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

(seatunnel) branch dependabot/npm_and_yarn/seatunnel-engine/seatunnel-engine-ui/vite-5.4.17 created (now 4dd60a67d7)

2025-04-09 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/npm_and_yarn/seatunnel-engine/seatunnel-engine-ui/vite-5.4.17 in repository https://gitbox.apache.org/repos/asf/seatunnel.git at 4dd60a67d7 Bump vite from 5.4.9 to 5.4.17 in

Re: [I] [Bug] [connectors-v2] 使用localfile 本地导入超过50万行的excel,速度越来越慢,提示oom [seatunnel]

2025-04-09 Thread via GitHub
misi1987107 commented on issue #8040: URL: https://github.com/apache/seatunnel/issues/8040#issuecomment-2788869982 How to reproduce this issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] [Bug] [Connector-V2] Schema evolution table names not handled correctly [seatunnel]

2025-04-09 Thread via GitHub
bwcxyk closed issue #9088: [Bug] [Connector-V2] Schema evolution table names not handled correctly URL: https://github.com/apache/seatunnel/issues/9088 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] [Feature][Transform] Support define sink column type [seatunnel]

2025-04-09 Thread via GitHub
hailin0 commented on PR #9114: URL: https://github.com/apache/seatunnel/pull/9114#issuecomment-2789145728 PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [PR] [improve] Adjust CosFile connector instantiation [seatunnel]

2025-04-09 Thread via GitHub
Hisoka-X commented on code in PR #9121: URL: https://github.com/apache/seatunnel/pull/9121#discussion_r2034861384 ## seatunnel-connectors-v2/connector-file/connector-file-cos/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/cos/source/CosFileSourceFactory.java: ## @@

Re: [PR] [bugfix] modify ObjectMapper configuration [seatunnel]

2025-04-09 Thread via GitHub
misi1987107 commented on PR #9126: URL: https://github.com/apache/seatunnel/pull/9126#issuecomment-2788865415 > Please add test case Test case has been added -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

(seatunnel) branch dependabot/npm_and_yarn/seatunnel-engine/seatunnel-engine-ui/vite-5.4.16 deleted (was d83f65208a)

2025-04-09 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/npm_and_yarn/seatunnel-engine/seatunnel-engine-ui/vite-5.4.16 in repository https://gitbox.apache.org/repos/asf/seatunnel.git was d83f65208a Bump vite from 5.4.9 to 5.4.16 in

Re: [PR] [feature][connectors-v2][ActiveMQ] add ActiveMQ source connector [seatunnel]

2025-04-09 Thread via GitHub
L-Gryps commented on PR #8581: URL: https://github.com/apache/seatunnel/pull/8581#issuecomment-2789249790 Please review. @Hisoka-X -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [I] Failed creating database DWD_LY in catalog Dameng [seatunnel]

2025-04-09 Thread via GitHub
tjhwk commented on issue #6537: URL: https://github.com/apache/seatunnel/issues/6537#issuecomment-2788075662 2.3.10这个版本依然存在这个问题,作为sink目标,database配置项应该怎么填写? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] postgrescdc compatible_debezium_json当数据库字段类型有NUMERIC(24)这种报错 [seatunnel]

2025-04-09 Thread via GitHub
Hisoka-X closed issue #9037: postgrescdc compatible_debezium_json当数据库字段类型有NUMERIC(24)这种报错 URL: https://github.com/apache/seatunnel/issues/9037 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[PR] [Feature] Add Workspace Support [seatunnel-web]

2025-04-09 Thread via GitHub
arshadmohammad opened a new pull request, #284: URL: https://github.com/apache/seatunnel-web/pull/284 Each resource is now associated with a workspaceId. If no workspace is specified during login, the default workspace will be used. ## Purpose of this pull request

Re: [PR] [improve] Adjust CosFile connector instantiation [seatunnel]

2025-04-09 Thread via GitHub
misi1987107 commented on code in PR #9121: URL: https://github.com/apache/seatunnel/pull/9121#discussion_r2035208562 ## seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/BaseFileSink.java: ## @@ -75,12 +7