Re: [I] [Feature][KafkaSource] The usage of StartMode in KafkaSource should have some Optimization [seatunnel]

2025-03-04 Thread via GitHub
litiliu commented on issue #8900: URL: https://github.com/apache/seatunnel/issues/8900#issuecomment-2700061738 I agree, we should remove the `earliest` and `latest` from the `start_mode` and use `group_offsets` as the default value. If use want to use the `earliest` or `latest` for the fi

[I] [Feature][KafkaSource] The usage of StartMode in KafkaSource should have some Optimization [seatunnel]

2025-03-04 Thread via GitHub
plaofchina opened a new issue, #8900: URL: https://github.com/apache/seatunnel/issues/8900 ### Search before asking - [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.

Re: [I] line 4: /bin/seatunnel.sh: No such file or directory [seatunnel]

2025-03-04 Thread via GitHub
qbmiller commented on issue #7066: URL: https://github.com/apache/seatunnel/issues/7066#issuecomment-2699631780 also need create user. dont use admin . sudo -u user -i xxx.sh . maybe change the path env -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [I] [Demos-Collection] Collecting Use Case Demos for Apache SeaTunnel [seatunnel]

2025-03-04 Thread via GitHub
davidzollo commented on issue #8388: URL: https://github.com/apache/seatunnel/issues/8388#issuecomment-2699585769 Maxcompute batch to PostgreSQL --- env { job.name="maxcompute-starrocks" job.mode="BATCH" parallelism = 3 } source { Maxcompute { accessId=

[I] [Bug] [KafkaSourceSplitReader] Conflict between commit_on_checkpoint and enable.auto.commit [seatunnel]

2025-03-04 Thread via GitHub
plaofchina opened a new issue, #8899: URL: https://github.com/apache/seatunnel/issues/8899 ### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happene

[PR] [Improve][Connector-V2] Random pick the starrocks fe address which can be connected [seatunnel]

2025-03-04 Thread via GitHub
xiaochen-zhou opened a new pull request, #8898: URL: https://github.com/apache/seatunnel/pull/8898 ### Purpose of this pull request Make sure the random and ha be supported at the same time. Random pick the address which can be connected. referring to the implementati

Re: [I] [Feature][Module Name]is sftpfile and ftpfile source connectors can add deletePolicy [seatunnel]

2025-03-04 Thread via GitHub
github-actions[bot] commented on issue #8590: URL: https://github.com/apache/seatunnel/issues/8590#issuecomment-2699349625 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

Re: [I] [Bug] [Sink-JDBC] Named parameters in SQL statement must not be empty [seatunnel]

2025-03-04 Thread via GitHub
github-actions[bot] closed issue #8530: [Bug] [Sink-JDBC] Named parameters in SQL statement must not be empty URL: https://github.com/apache/seatunnel/issues/8530 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [I] [Bug][Connector-V2][Assert] AssertSinkWriter supports multiple concurrent operations [seatunnel]

2025-03-04 Thread via GitHub
github-actions[bot] closed issue #8357: [Bug][Connector-V2][Assert] AssertSinkWriter supports multiple concurrent operations URL: https://github.com/apache/seatunnel/issues/8357 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] [Feature][Flink] Optimization of Flink Source Busy Problem [seatunnel]

2025-03-04 Thread via GitHub
github-actions[bot] closed issue #8358: [Feature][Flink] Optimization of Flink Source Busy Problem URL: https://github.com/apache/seatunnel/issues/8358 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [I] [Bug] [Sink-JDBC] Named parameters in SQL statement must not be empty [seatunnel]

2025-03-04 Thread via GitHub
github-actions[bot] commented on issue #8530: URL: https://github.com/apache/seatunnel/issues/8530#issuecomment-2699349684 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is a

Re: [I] [Feature][Core] Support Flink 1.20.0 [seatunnel]

2025-03-04 Thread via GitHub
github-actions[bot] closed issue #8531: [Feature][Core] Support Flink 1.20.0 URL: https://github.com/apache/seatunnel/issues/8531 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] [Feature][Core] Support Flink 1.20.0 [seatunnel]

2025-03-04 Thread via GitHub
github-actions[bot] commented on issue #8531: URL: https://github.com/apache/seatunnel/issues/8531#issuecomment-2699349654 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is a

Re: [I] [Feature][Flink] Optimization of Flink Source Busy Problem [seatunnel]

2025-03-04 Thread via GitHub
github-actions[bot] commented on issue #8358: URL: https://github.com/apache/seatunnel/issues/8358#issuecomment-2699349737 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is a

Re: [I] [Bug][Connector-V2][Assert] AssertSinkWriter supports multiple concurrent operations [seatunnel]

2025-03-04 Thread via GitHub
github-actions[bot] commented on issue #8357: URL: https://github.com/apache/seatunnel/issues/8357#issuecomment-2699349761 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is a

Re: [PR] [Feature][Connector-V2] Add support for Time type in paimon connector [seatunnel]

2025-03-04 Thread via GitHub
xiaochen-zhou commented on PR #8880: URL: https://github.com/apache/seatunnel/pull/8880#issuecomment-2699305836 > could you add Time type in these e2e case ? > > ![image](https://private-user-images.githubusercontent.com/39961809/418622327-c7170444-f501-4f14-ba04-1e12e135c378.png?jwt=

Re: [I] [Bug] 基于主键批量同步Oracle至MySQL时,Average Read Count与Average Write Count始终为0 [seatunnel]

2025-03-04 Thread via GitHub
chenyz1984 commented on issue #8702: URL: https://github.com/apache/seatunnel/issues/8702#issuecomment-2699253665 > 如何解决始终为0 还没解决,技术社区咨询,也没人回应。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] [Feature][Iceberg] Support merge fragmented files [seatunnel]

2025-03-04 Thread via GitHub
sunxiaojian commented on code in PR #8840: URL: https://github.com/apache/seatunnel/pull/8840#discussion_r1979262469 ## seatunnel-connectors-v2/connector-iceberg/src/main/java/org/apache/seatunnel/connectors/seatunnel/iceberg/compaction/IcebergCompactionHandler.java: ## @@ -0,0

Re: [I] [Bug] [Clickhouse to localfile] export Data rows error,need 1000,000 but actual 2000,000 [seatunnel]

2025-03-04 Thread via GitHub
Hisoka-X commented on issue #8803: URL: https://github.com/apache/seatunnel/issues/8803#issuecomment-2697363422 > Hi, this is a bug. https://github.com/apache/seatunnel/blob/dev/seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/so

Re: [I] [Umbrella] unified parameter class and refactor connector creation logic [seatunnel]

2025-03-04 Thread via GitHub
xdu-chenrj commented on issue #8576: URL: https://github.com/apache/seatunnel/issues/8576#issuecomment-2697170386 I would like to start with email. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] [Feature][Iceberg] Support merge fragmented files [seatunnel]

2025-03-04 Thread via GitHub
hailin0 commented on code in PR #8840: URL: https://github.com/apache/seatunnel/pull/8840#discussion_r1979169259 ## seatunnel-connectors-v2/connector-iceberg/src/main/java/org/apache/seatunnel/connectors/seatunnel/iceberg/compaction/IcebergCompactionHandler.java: ## @@ -0,0 +1,2

Re: [PR] [Feature][Iceberg] Support merge fragmented files [seatunnel]

2025-03-04 Thread via GitHub
hailin0 commented on code in PR #8840: URL: https://github.com/apache/seatunnel/pull/8840#discussion_r1979169259 ## seatunnel-connectors-v2/connector-iceberg/src/main/java/org/apache/seatunnel/connectors/seatunnel/iceberg/compaction/IcebergCompactionHandler.java: ## @@ -0,0 +1,2

Re: [I] [Bug] [Clickhouse to localfile] export Data rows error,need 1000,000 but actual 2000,000 [seatunnel]

2025-03-04 Thread via GitHub
liunaijie commented on issue #8803: URL: https://github.com/apache/seatunnel/issues/8803#issuecomment-2697067552 Hi, this is a bug. https://github.com/apache/seatunnel/blob/dev/seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse

[I] [Bug] [Source] NoSuchMethodError: org.apache.seatunnel.format.json.JsonDeserializationSchema [seatunnel]

2025-03-04 Thread via GitHub
JunjianS opened a new issue, #8897: URL: https://github.com/apache/seatunnel/issues/8897 ### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened

Re: [PR] 2.3.9-fixes [seatunnel]

2025-03-04 Thread via GitHub
pj001 closed pull request #8891: 2.3.9-fixes URL: https://github.com/apache/seatunnel/pull/8891 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-

Re: [PR] [Improve][connector-hive] Improved hive file allocation algorithm for subtasks [seatunnel]

2025-03-04 Thread via GitHub
liunaijie merged PR #8876: URL: https://github.com/apache/seatunnel/pull/8876 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatun

(seatunnel) branch dev updated (3cfe8c12b9 -> 89d1878ade)

2025-03-04 Thread jarvis
This is an automated email from the ASF dual-hosted git repository. jarvis pushed a change to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git from 3cfe8c12b9 [Improve] sink mongodb schema is not required (#8887) add 89d1878ade [Improve][connector-hive] Improve

Re: [PR] [hotfix][redis] fix npe cause by null host parameter [seatunnel]

2025-03-04 Thread via GitHub
liunaijie commented on code in PR #8881: URL: https://github.com/apache/seatunnel/pull/8881#discussion_r1979004527 ## seatunnel-connectors-v2/connector-redis/src/main/java/org/apache/seatunnel/connectors/seatunnel/redis/config/RedisParameters.java: ## @@ -46,7 +46,7 @@ @Slf4j

(seatunnel) branch dev updated: [Improve] sink mongodb schema is not required (#8887)

2025-03-04 Thread jarvis
This is an automated email from the ASF dual-hosted git repository. jarvis pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new 3cfe8c12b9 [Improve] sink mongodb schema is not req

Re: [PR] [Improve] sink mongodb schema is not required [seatunnel]

2025-03-04 Thread via GitHub
liunaijie merged PR #8887: URL: https://github.com/apache/seatunnel/pull/8887 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatun

Re: [PR] 2.3.9-fixes [seatunnel]

2025-03-04 Thread via GitHub
liunaijie commented on code in PR #8891: URL: https://github.com/apache/seatunnel/pull/8891#discussion_r1978984876 ## .gitignore: ## @@ -48,8 +48,6 @@ test.conf spark-warehouse *.flattened-pom.xml -seatunnel-examples - Review Comment: Do not update this file #

[PR] [Fix][Connector-V2] Fix maxcompute read with partition spec [seatunnel]

2025-03-04 Thread via GitHub
Hisoka-X opened a new pull request, #8896: URL: https://github.com/apache/seatunnel/pull/8896 ### Purpose of this pull request ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? ### Check list

[PR] [Feature][elasticsearch-connector] support elasticsearch sql source [seatunnel]

2025-03-04 Thread via GitHub
CosmosNi opened a new pull request, #8895: URL: https://github.com/apache/seatunnel/pull/8895 close #8894 ### Purpose of this pull request ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? #

Re: [I] How PostgreSQL-CDC uses heartbeat.action.query to reduce WAL disk space consumption [seatunnel]

2025-03-04 Thread via GitHub
huangyutongs commented on issue #8762: URL: https://github.com/apache/seatunnel/issues/8762#issuecomment-2696551940 Is anyone there? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [I] [Bug] [Checkoint] checkpoint fail in 2.3.6 [seatunnel]

2025-03-04 Thread via GitHub
acttk commented on issue #7339: URL: https://github.com/apache/seatunnel/issues/7339#issuecomment-2696638841 如何解决的兄弟 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] [Fix][MongoDB] The Long type cannot handle string values in scientific notation. #8775 [seatunnel]

2025-03-04 Thread via GitHub
qifanlili commented on PR #8783: URL: https://github.com/apache/seatunnel/pull/8783#issuecomment-2696617795 > Thanks @qifanlili ! Could you add a test case in https://github.com/apache/seatunnel/tree/c1b2675ab0826bc90a16766b1811f9f3dc4c2c0c/seatunnel-connectors-v2/connector-mongodb/src/test/

[I] [Feature][elasticsearch-connector] support elasticsearch sql source [seatunnel]

2025-03-04 Thread via GitHub
CosmosNi opened a new issue, #8894: URL: https://github.com/apache/seatunnel/issues/8894 ### Search before asking - [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.

Re: [PR] [Feature][Paimon] Customize the hadoop user [seatunnel]

2025-03-04 Thread via GitHub
hawk9821 commented on PR #: URL: https://github.com/apache/seatunnel/pull/#issuecomment-2696582905 > Does hadoop has own key to set read user? the key is `HADOOP_USER_NAME` ![image](https://github.com/user-attachments/assets/4883d3f4-dd7d-4954-be94-7ceb9cfef4e1) --

Re: [I] [Bug] 基于主键批量同步Oracle至MySQL时,Average Read Count与Average Write Count始终为0 [seatunnel]

2025-03-04 Thread via GitHub
acttk commented on issue #8702: URL: https://github.com/apache/seatunnel/issues/8702#issuecomment-2696569007 如何解决始终为0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] [feature][connectors-v2][ActiveMQ] add ActiveMQ source connector [seatunnel]

2025-03-04 Thread via GitHub
L-Gryps commented on PR #8581: URL: https://github.com/apache/seatunnel/pull/8581#issuecomment-2696533210 Please review. I added message acknowledge mechanism @Hisoka-X -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use