Re: [PR] [Feature][Connector-Paimon] Support dynamic bucket splitting improves Paimon writing efficiency [seatunnel]

2024-08-29 Thread via GitHub
hawk9821 commented on PR #7335: URL: https://github.com/apache/seatunnel/pull/7335#issuecomment-2316870708 > Why are there so many file changes. Maybe you're having some problems with your git operations. Please reopen a pr which must only has your commits. Then you can link this pr in new

Re: [PR] [Feature][Connector-Paimon] Support dynamic bucket splitting improves Paimon writing efficiency [seatunnel]

2024-08-29 Thread via GitHub
hawk9821 commented on PR #7335: URL: https://github.com/apache/seatunnel/pull/7335#issuecomment-2316878588 > Thinks @hawk9821 . Good job. I think your e2e case needs to be added to the case of multi-parallelism, the current case is all single parallelism. In this way, we can effectively ver

Re: [PR] [Feature][Connector-V2] [Hudi]Add hudi sink connector [seatunnel]

2024-08-29 Thread via GitHub
liugddx commented on PR #4405: URL: https://github.com/apache/seatunnel/pull/4405#issuecomment-2317066220 https://github.com/leesf PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[I] [Bug] [cup] if task starting,cpu will 100% [seatunnel]

2024-08-29 Thread via GitHub
13813586515 opened a new issue, #7521: URL: https://github.com/apache/seatunnel/issues/7521 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happen

Re: [PR] [Feature][Connector-V2][Hbase] implement hbase catalog [seatunnel]

2024-08-29 Thread via GitHub
zhangshenghang commented on code in PR #7516: URL: https://github.com/apache/seatunnel/pull/7516#discussion_r1736115265 ## seatunnel-e2e/seatunnel-connector-v2-e2e/connector-hbase-e2e/src/test/java/org/apache/seatunnel/e2e/connector/hbase/HbaseIT.java: ## @@ -50,9 +53,13 @@ @

[PR] [Feature][Core] Added rest-api for batch start and stop [seatunnel]

2024-08-29 Thread via GitHub
zhangshenghang opened a new pull request, #7522: URL: https://github.com/apache/seatunnel/pull/7522 ### Purpose of this pull request - resolve #7511 ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? Local and e2e

Re: [PR] [Feature][Rabbitmq] Allow configuration of queue durability and deletion policy [seatunnel]

2024-08-29 Thread via GitHub
zqr10159 commented on PR #7365: URL: https://github.com/apache/seatunnel/pull/7365#issuecomment-2317552613 @Hisoka-X All done, Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] [Feature][Rabbitmq] Allow configuration of queue durability and deletion policy [seatunnel]

2024-08-29 Thread via GitHub
hailin0 closed issue #7363: [Feature][Rabbitmq] Allow configuration of queue durability and deletion policy URL: https://github.com/apache/seatunnel/issues/7363 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] [Feature][Rabbitmq] Allow configuration of queue durability and deletion policy [seatunnel]

2024-08-29 Thread via GitHub
hailin0 merged PR #7365: URL: https://github.com/apache/seatunnel/pull/7365 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunne

(seatunnel) branch dev updated: [Feature][Rabbitmq] Allow configuration of queue durability and deletion policy (#7365)

2024-08-29 Thread wanghailin
This is an automated email from the ASF dual-hosted git repository. wanghailin pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new aabfc8eb78 [Feature][Rabbitmq] Allow configurat

Re: [PR] [Feature][Connector-V2] Support typesense connector [seatunnel]

2024-08-29 Thread via GitHub
hailin0 merged PR #7450: URL: https://github.com/apache/seatunnel/pull/7450 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunne

[I] [Feature][Engine] Support Seatunnel on Yarn [seatunnel]

2024-08-29 Thread via GitHub
FlechazoW opened a new issue, #7523: URL: https://github.com/apache/seatunnel/issues/7523 ### Search before asking - [X] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.

[PR] [Improve][Transform] Ignore error with jsonpath transform [seatunnel]

2024-08-29 Thread via GitHub
hailin0 opened a new pull request, #7524: URL: https://github.com/apache/seatunnel/pull/7524 ### Purpose of this pull request [Improve][Transform] Ignore error with jsonpath transform ### Does this PR introduce _any_ user-facing change? Add new config item

Re: [PR] [Improve][Connector-V2] Remove hard code iceberg table format version [seatunnel]

2024-08-29 Thread via GitHub
hailin0 merged PR #7500: URL: https://github.com/apache/seatunnel/pull/7500 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunne

(seatunnel) branch dev updated: [Improve][Connector-V2] Remove hard code iceberg table format version (#7500)

2024-08-29 Thread wanghailin
This is an automated email from the ASF dual-hosted git repository. wanghailin pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new f49b263e65 [Improve][Connector-V2] Remove hard

[I] [Improvement] [Seatunnel-web] Add API to get and update job definition. [seatunnel]

2024-08-29 Thread via GitHub
arshadmohammad opened a new issue, #7525: URL: https://github.com/apache/seatunnel/issues/7525 ### Search before asking - [X] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.

[PR] [Improvement] [Seatunnel-web] Add API to get and update job definition. [seatunnel-web]

2024-08-29 Thread via GitHub
arshadmohammad opened a new pull request, #198: URL: https://github.com/apache/seatunnel-web/pull/198 As part of https://github.com/apache/seatunnel-web/commit/f45dacd11759bcf0fb0ad363b334d0a00af5e69a we have added single API to create a job. There should be corresponding get and update AP

Re: [I] [Improvement] [Seatunnel-web] Add API to get and update job definition. [seatunnel]

2024-08-29 Thread via GitHub
arshadmohammad commented on issue #7525: URL: https://github.com/apache/seatunnel/issues/7525#issuecomment-2318043517 Raised PR https://github.com/apache/seatunnel-web/pull/198 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] [Improvement] [Seatunnel-web] Add API to get and update job definition. [seatunnel-web]

2024-08-29 Thread via GitHub
arshadmohammad commented on PR #198: URL: https://github.com/apache/seatunnel-web/pull/198#issuecomment-2318040041 Tested the newly added API though test case and run all the integration tests. ![image](https://github.com/user-attachments/assets/18e9014d-2613-404c-9122-bea22b3a8aaf)

[I] [Feature][Transform] Add embedding transform [seatunnel]

2024-08-29 Thread via GitHub
corgy-w opened a new issue, #7526: URL: https://github.com/apache/seatunnel/issues/7526 ### Search before asking - [X] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement. #

Re: [I] [Bug] [Connector-V2] fix jdbc-multi-metrics bug [seatunnel]

2024-08-29 Thread via GitHub
corgy-w closed issue #7483: [Bug] [Connector-V2] fix jdbc-multi-metrics bug URL: https://github.com/apache/seatunnel/issues/7483 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [Improvement] [Seatunnel-web] Add API to get job execution status. [seatunnel-web]

2024-08-29 Thread via GitHub
arshadmohammad commented on PR #197: URL: https://github.com/apache/seatunnel-web/pull/197#issuecomment-2318055644 Thanks @Hisoka-X for reviewing and merging the PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [I] [Improvement] [Seatunnel-web] Add API to get job execution status. [seatunnel]

2024-08-29 Thread via GitHub
arshadmohammad closed issue #7518: [Improvement] [Seatunnel-web] Add API to get job execution status. URL: https://github.com/apache/seatunnel/issues/7518 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] [Improvement] [Seatunnel-web] Add API to get job execution status. [seatunnel]

2024-08-29 Thread via GitHub
arshadmohammad commented on issue #7518: URL: https://github.com/apache/seatunnel/issues/7518#issuecomment-2318060597 PR is merged https://github.com/apache/seatunnel-web/commit/c0880eeefd1b3f2242c3d42b799ddd5f15c5ce47 -- This is an automated message from the Apache Git Service. To respon

Re: [I] 把influxdb里面的数据同步到doris [seatunnel]

2024-08-29 Thread via GitHub
github-actions[bot] commented on issue #7250: URL: https://github.com/apache/seatunnel/issues/7250#issuecomment-2319540742 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is a

Re: [I] 把influxdb里面的数据同步到doris [seatunnel]

2024-08-29 Thread via GitHub
github-actions[bot] closed issue #7250: 把influxdb里面的数据同步到doris URL: https://github.com/apache/seatunnel/issues/7250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [I] [Feature][Hudi Sink] Hudi sink supports catalog and typeconverter features. [seatunnel]

2024-08-29 Thread via GitHub
github-actions[bot] closed issue #7249: [Feature][Hudi Sink] Hudi sink supports catalog and typeconverter features. URL: https://github.com/apache/seatunnel/issues/7249 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] [Feature]Parquet version upgrade to support LZ4_RAW compression format [seatunnel]

2024-08-29 Thread via GitHub
github-actions[bot] closed issue #7223: [Feature]Parquet version upgrade to support LZ4_RAW compression format URL: https://github.com/apache/seatunnel/issues/7223 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [I] [Feature][Hudi Sink] Hudi sink supports catalog and typeconverter features. [seatunnel]

2024-08-29 Thread via GitHub
github-actions[bot] commented on issue #7249: URL: https://github.com/apache/seatunnel/issues/7249#issuecomment-2319540788 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is a

Re: [I] [Feature]Parquet version upgrade to support LZ4_RAW compression format [seatunnel]

2024-08-29 Thread via GitHub
github-actions[bot] commented on issue #7223: URL: https://github.com/apache/seatunnel/issues/7223#issuecomment-2319540823 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is a

Re: [PR] [WIP][feature][spark] Support streaming [seatunnel]

2024-08-29 Thread via GitHub
Carl-Zhou-CN commented on PR #7476: URL: https://github.com/apache/seatunnel/pull/7476#issuecomment-2319646433 > > > https://github.com/apache/seatunnel/blob/1bba72385b6797dc5edd96fa5951376d0594e633/seatunnel-translation/seatunnel-translation-spark/seatunnel-translation-spark-3.3/src/main/ja

Re: [PR] [WIP][feature][spark] Support streaming [seatunnel]

2024-08-29 Thread via GitHub
Carl-Zhou-CN commented on PR #7476: URL: https://github.com/apache/seatunnel/pull/7476#issuecomment-2319648997 > The checkpoint space like this: > > ```shell > ./ > ├── commits > │   ├ > │   ├── 10 > │   ├── 11 > │   ├── 12 > │   ├── 13 > │   ├── 14 >

Re: [PR] [WIP][feature][spark] Support streaming [seatunnel]

2024-08-29 Thread via GitHub
Carl-Zhou-CN commented on PR #7476: URL: https://github.com/apache/seatunnel/pull/7476#issuecomment-2319657631 > > > https://github.com/apache/seatunnel/blob/1bba72385b6797dc5edd96fa5951376d0594e633/seatunnel-translation/seatunnel-translation-spark/seatunnel-translation-spark-3.3/src/main/ja

[PR] [Hotfix][Connector-V2] Fixed lost data precision for decimal data types [seatunnel]

2024-08-29 Thread via GitHub
dailai opened a new pull request, #7527: URL: https://github.com/apache/seatunnel/pull/7527 ### Purpose of this pull request Fix the problem that the data type precision of decimal at the sink side is greater than the precision at the source side, the data is written incorrectly.

Re: [PR] [Feature][Connector-V2]Support tidb cdc connector source [seatunnel]

2024-08-29 Thread via GitHub
dailai commented on PR #7477: URL: https://github.com/apache/seatunnel/pull/7477#issuecomment-2319680903 Please fix the ci. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] [Bug] [Mongo] The column type is , but a null value is being written into it [seatunnel]

2024-08-29 Thread via GitHub
WilliaJing commented on issue #6884: URL: https://github.com/apache/seatunnel/issues/6884#issuecomment-2319696347 yes,I have also found it. The official new version has been fixed. Thank you for your reply😄 @Light-Towers -- This is an automated message from the Apache Git Service. To res

Re: [I] [Feature]Parquet version upgrade to support LZ4_RAW compression format [seatunnel]

2024-08-29 Thread via GitHub
fcb-xiaobo commented on issue #7223: URL: https://github.com/apache/seatunnel/issues/7223#issuecomment-2319710841 please assign to me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[I] [Bug] [connector-v2] Dependency Conflict [seatunnel]

2024-08-29 Thread via GitHub
Xuzhengz opened a new issue, #7528: URL: https://github.com/apache/seatunnel/issues/7528 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened

Re: [PR] [Improvement] [Seatunnel-web] Add API to get and update job definition. [seatunnel-web]

2024-08-29 Thread via GitHub
Hisoka-X merged PR #198: URL: https://github.com/apache/seatunnel-web/pull/198 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatu

Re: [I] [Improvement] [Seatunnel-web] Add API to get and update job definition. [seatunnel]

2024-08-29 Thread via GitHub
Hisoka-X closed issue #7525: [Improvement] [Seatunnel-web] Add API to get and update job definition. URL: https://github.com/apache/seatunnel/issues/7525 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

(seatunnel-web) branch main updated: [Improvement] [Seatunnel-web] Add API to get and update job definition. (#198)

2024-08-29 Thread fanjia
This is an automated email from the ASF dual-hosted git repository. fanjia pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/seatunnel-web.git The following commit(s) were added to refs/heads/main by this push: new 30ed42fd [Improvement] [Seatunnel-web] Add AP

Re: [I] [Improvement] [Seatunnel-web] Add API to get and update job definition. [seatunnel]

2024-08-29 Thread via GitHub
Hisoka-X commented on issue #7525: URL: https://github.com/apache/seatunnel/issues/7525#issuecomment-2319745829 Closed by https://github.com/apache/seatunnel-web/pull/198 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] [Feature][Connector Hive] support hive savemode [seatunnel]

2024-08-29 Thread via GitHub
liunaijie commented on PR #6842: URL: https://github.com/apache/seatunnel/pull/6842#issuecomment-2319850378 > @liunaijie Does sink support writing to specified partitions? Hi @wuchunfu this pr only add hive savemode function. you mentioned `write to specified partition` it's impleme

[PR] [Feature][Clickhouse] Support multi-table source read [seatunnel]

2024-08-29 Thread via GitHub
zhilinli123 opened a new pull request, #7529: URL: https://github.com/apache/seatunnel/pull/7529 ### Purpose of this pull request ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? ### Check list

Re: [I] [Bug] [connector-v2] Dependency Conflict [seatunnel]

2024-08-29 Thread via GitHub
Xuzhengz commented on issue #7528: URL: https://github.com/apache/seatunnel/issues/7528#issuecomment-2319943728 Reason for S3 plugin error: The guava version in the Hive driver is lower, while the guava version in the S3 plugin is higher, but now the driver dependencies are all in lib. How

Re: [PR] [Feature][Core] Added rest-api for batch start and stop [seatunnel]

2024-08-29 Thread via GitHub
Hisoka-X commented on code in PR #7522: URL: https://github.com/apache/seatunnel/pull/7522#discussion_r1737866030 ## seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/rest/RestHttpPostCommandProcessor.java: ## @@ -103,11 +112,41 @@ private

Re: [PR] [Hotfix][Connector-V2] Fixed precision problem of decimal data types for paimon sink [seatunnel]

2024-08-29 Thread via GitHub
wuchunfu merged PR #7527: URL: https://github.com/apache/seatunnel/pull/7527 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunn

(seatunnel) branch dev updated: [Hotfix][Connector-V2] Fixed lost data precision for decimal data types (#7527)

2024-08-29 Thread wuchunfu
This is an automated email from the ASF dual-hosted git repository. wuchunfu pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new df210ea73d [Hotfix][Connector-V2] Fixed lost data

Re: [I] [Feature][Transform] Support JOIN operations in transform and create complex graph ETL [seatunnel]

2024-08-29 Thread via GitHub
hexnn commented on issue #7266: URL: https://github.com/apache/seatunnel/issues/7266#issuecomment-2320134541 你应该是在找这个:https://github.com/hexnn/stark -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [I] [Feature][Core] The feasibility of seatunnel as an underlying support for an ETL tool(seatunnel作为ETL工具的底层支持的可行性) [seatunnel]

2024-08-29 Thread via GitHub
hexnn commented on issue #6913: URL: https://github.com/apache/seatunnel/issues/6913#issuecomment-2320137757 这个支持多表join和groupby:https://github.com/hexnn/stark -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[PR] [Improve][Connector-v2] Support mysql 8.1/8.2/8.3 for jdbc [seatunnel]

2024-08-29 Thread via GitHub
dailai opened a new pull request, #7530: URL: https://github.com/apache/seatunnel/pull/7530 ### Purpose of this pull request Solve the problem #7514 We should support this version: https://github.com/user-attachments/assets/c5351e0a-8a1b-4a95-82b7-92c771a99e6a";>

Re: [PR] [WIP][feature][spark] Support streaming [seatunnel]

2024-08-29 Thread via GitHub
CheneyYin commented on PR #7476: URL: https://github.com/apache/seatunnel/pull/7476#issuecomment-2320207764 > > > > https://github.com/apache/seatunnel/blob/1bba72385b6797dc5edd96fa5951376d0594e633/seatunnel-translation/seatunnel-translation-spark/seatunnel-translation-spark-3.3/src/main/jav