Re: [I] [Bug] [Module Name] Hive2CK using CK File generate data_local.log slow [seatunnel]

2023-11-08 Thread via GitHub
Hisoka-X commented on issue #5819: URL: https://github.com/apache/seatunnel/issues/5819#issuecomment-1803323147 Sure! Looking forward your PR! @viverlxl -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] [Bug] [Module Name] Hive2CK using CK File generate data_local.log slow [seatunnel]

2023-11-08 Thread via GitHub
viverlxl commented on issue #5819: URL: https://github.com/apache/seatunnel/issues/5819#issuecomment-1803304601 @Hisoka-X can i fix this issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] [Bug] [Module Name] Hive2CK using CK File generate data_local.log slow [seatunnel]

2023-11-08 Thread via GitHub
viverlxl commented on issue #5819: URL: https://github.com/apache/seatunnel/issues/5819#issuecomment-1803300304 https://github.com/apache/seatunnel/assets/35752202/ac2ef499-3eef-483a-aed5-983aca320144";> 原代码和修改后的代码做对比 -- This is an automated message from the Apache Git Service. To respo

Re: [I] [Bug] [Module Name] Hive2CK using CK File generate data_local.log slow [seatunnel]

2023-11-08 Thread via GitHub
viverlxl commented on issue #5819: URL: https://github.com/apache/seatunnel/issues/5819#issuecomment-1803298048 use mmap buffer can solve this issue https://github.com/apache/seatunnel/assets/35752202/13086f6e-ee47-4cea-83dd-05081a7d8de0";> 写2.3kw到data_local.log, 从原来的5小时变成4分钟。表象来看是一

Re: [I] [Bug]clickhouse cdc sink ClickhouseConnectorException [seatunnel]

2023-11-08 Thread via GitHub
happyboy1024 commented on issue #5818: URL: https://github.com/apache/seatunnel/issues/5818#issuecomment-1803216588 This is fixed by https://github.com/apache/seatunnel/pull/5448 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] [Feature][Transform-V2] Support SqlTransform Not Like Expression [seatunnel]

2023-11-08 Thread via GitHub
zhilinli123 commented on PR #5768: URL: https://github.com/apache/seatunnel/pull/5768#issuecomment-1803194619 PTAL: @Hisoka-X @EricJoy2048 @hailin0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [I] [Bug] [Module Name] Hive2CK using CK File generate data_local.log slow [seatunnel]

2023-11-08 Thread via GitHub
viverlxl commented on issue #5819: URL: https://github.com/apache/seatunnel/issues/5819#issuecomment-1803159736 https://github.com/apache/seatunnel/assets/35752202/c2a21770-f944-4c1d-8047-1c3411d4bf34";> 如图中的每条数据一个mmap操作导致写入性能不太好 -- This is an automated message from the Apache Git S

[PR] [Test] Fix ConnectorSpecificationCheckTest not work [seatunnel]

2023-11-08 Thread via GitHub
Hisoka-X opened a new pull request, #5820: URL: https://github.com/apache/seatunnel/pull/5820 ### Purpose of this pull request ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? ### Check list * [

[I] [Bug] [Module Name] Hive2CK using CK File generate data_local.log slow [seatunnel]

2023-11-08 Thread via GitHub
viverlxl opened a new issue, #5819: URL: https://github.com/apache/seatunnel/issues/5819 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened

[I] [Bug]clickhouse cdc sink ClickhouseConnectorException [seatunnel]

2023-11-08 Thread via GitHub
chase-cai-huan opened a new issue, #5818: URL: https://github.com/apache/seatunnel/issues/5818 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What hap

[I] [Bug] [mysql-cdc] mysql cdc 任务提交异常退出不会自动断开连接 [seatunnel]

2023-11-08 Thread via GitHub
chase-cai-huan opened a new issue, #5817: URL: https://github.com/apache/seatunnel/issues/5817 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What hap

Re: [I] How to configure multiple MySQL sources with identical table structures to synchronize to one sink? [seatunnel]

2023-11-08 Thread via GitHub
github-actions[bot] commented on issue #5596: URL: https://github.com/apache/seatunnel/issues/5596#issuecomment-1802969154 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

Re: [I] Seatunel supports zeppelin data analysis tool [seatunnel]

2023-11-08 Thread via GitHub
github-actions[bot] commented on issue #5599: URL: https://github.com/apache/seatunnel/issues/5599#issuecomment-1802969131 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

Re: [I] [Feature][Rest API] When get job information based on jobId, job exception information is returned. [seatunnel]

2023-11-08 Thread via GitHub
github-actions[bot] commented on issue #5600: URL: https://github.com/apache/seatunnel/issues/5600#issuecomment-1802969106 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

Re: [I] [Bug] [connector-v2] jdbc source postgresql Doesn't support Postgres type 'unknown' yet [seatunnel]

2023-11-08 Thread via GitHub
github-actions[bot] commented on issue #5603: URL: https://github.com/apache/seatunnel/issues/5603#issuecomment-1802969027 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

Re: [I] [Feature][MqttSource] Want to support mqtt protocol [seatunnel]

2023-11-08 Thread via GitHub
github-actions[bot] commented on issue #5602: URL: https://github.com/apache/seatunnel/issues/5602#issuecomment-1802969049 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

Re: [I] [Bug] [SqlserverCDC Source] Sqlserver CDC Doesn't support SQLSERVER type 'GEOGRAPHY' [seatunnel]

2023-11-08 Thread via GitHub
github-actions[bot] commented on issue #5601: URL: https://github.com/apache/seatunnel/issues/5601#issuecomment-1802969072 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

Re: [PR] [Feature] support avro format [seatunnel]

2023-11-08 Thread via GitHub
zhilinli123 commented on code in PR #5084: URL: https://github.com/apache/seatunnel/pull/5084#discussion_r1386808552 ## seatunnel-e2e/seatunnel-connector-v2-e2e/connector-kafka-e2e/src/test/java/org/apache/seatunnel/e2e/connector/kafka/KafkaIT.java: ## @@ -292,6 +293,30 @@ publi

Re: [I] [Bug] [Module Name] Hive Source read_partitions option 当配置的分区不在表文件目录时会被throw 异常提早退出 [seatunnel]

2023-11-08 Thread via GitHub
hailin0 commented on issue #5811: URL: https://github.com/apache/seatunnel/issues/5811#issuecomment-1801984766 this pr fixed https://github.com/apache/seatunnel/pull/5591 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] [Bug] [Module Name] Hive Source read_partitions option 当配置的分区不在表文件目录时会被throw 异常提早退出 [seatunnel]

2023-11-08 Thread via GitHub
Carl-Zhou-CN commented on issue #5811: URL: https://github.com/apache/seatunnel/issues/5811#issuecomment-1801860036 @Hisoka-X I'd like to try it. Please assign it to me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[PR] [Feature][Connector-V2] Support TableSourceFactory/TableSinkFactory on http #5807 [seatunnel]

2023-11-08 Thread via GitHub
jacky opened a new pull request, #5816: URL: https://github.com/apache/seatunnel/pull/5816 ### Purpose of this pull request ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? improve current e2e test

Re: [I] 2.3.2启动错误java.lang.NoClassDefFoundError: com/sun/jersey/client/impl/CopyOnWriteHashMap [seatunnel]

2023-11-08 Thread via GitHub
vleyong commented on issue #5334: URL: https://github.com/apache/seatunnel/issues/5334#issuecomment-1801781300 > Put this [file](https://repo.maven.apache.org/maven2/org/apache/seatunnel/seatunnel-hadoop3-3.1.4-uber/2.3.2/seatunnel-hadoop3-3.1.4-uber-2.3.2-optional.jar) to $seatunnel/lib

(seatunnel) branch dev updated: [Improve][MongoDB] Implement TableSourceFactory to create mongodb source (#5813)

2023-11-08 Thread fanjia
This is an automated email from the ASF dual-hosted git repository. fanjia pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new 59cccb6097 [Improve][MongoDB] Implement TableSource

Re: [PR] [Improve] Code clean for SeaTunnel transform [seatunnel]

2023-11-08 Thread via GitHub
Hisoka-X commented on code in PR #5810: URL: https://github.com/apache/seatunnel/pull/5810#discussion_r1386469725 ## seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLEngine.java: ## @@ -244,9 +244,6 @@ private Object[] project(Object[] inputFi

Re: [PR] [Improve] Code clean for SeaTunnel transform [seatunnel]

2023-11-08 Thread via GitHub
hailin0 commented on code in PR #5810: URL: https://github.com/apache/seatunnel/pull/5810#discussion_r1386468740 ## seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/ZetaSQLEngine.java: ## @@ -244,9 +244,6 @@ private Object[] project(Object[] inputFie

Re: [PR] [Improve][MongoDB] Implement TableSourceFactory to create mongodb source [seatunnel]

2023-11-08 Thread via GitHub
Hisoka-X merged PR #5813: URL: https://github.com/apache/seatunnel/pull/5813 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunn

(seatunnel) branch dev updated: [Improve][format] Using number format for Decimal type in `seatunnel-format-compatible-debezium-json` (#5803)

2023-11-08 Thread fanjia
This is an automated email from the ASF dual-hosted git repository. fanjia pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new 7b3a5d3c59 [Improve][format] Using number format fo

Re: [PR] [Improve][format] Using number format for Decimal type in `seatunnel-format-compatible-debezium-json` [seatunnel]

2023-11-08 Thread via GitHub
Hisoka-X merged PR #5803: URL: https://github.com/apache/seatunnel/pull/5803 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunn

[PR] [Improve] Add common error for transform [seatunnel]

2023-11-08 Thread via GitHub
Hisoka-X opened a new pull request, #5815: URL: https://github.com/apache/seatunnel/pull/5815 ### Purpose of this pull request After introduce new error define rule in #5793 . This PR rewrite part error of transform to `INPUT_FIELD(S)_NOT_FOUND`. ### Does this PR in

Re: [PR] [Improve][Common] Introduce new error define rule [seatunnel]

2023-11-08 Thread via GitHub
EricJoy2048 merged PR #5793: URL: https://github.com/apache/seatunnel/pull/5793 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seat

[PR] [Fix] Fix read from Oracle Date type value lose time [seatunnel]

2023-11-08 Thread via GitHub
Hisoka-X opened a new pull request, #5814: URL: https://github.com/apache/seatunnel/pull/5814 ### Purpose of this pull request This PR fix read from Oracle Date type value lose time part value. It cause by wrong type convert, before this PR, SeaTunnel convert Date type to SeaTunne

Re: [PR] [Feature] add dockerfile [seatunnel]

2023-11-08 Thread via GitHub
liunaijie commented on PR #5526: URL: https://github.com/apache/seatunnel/pull/5526#issuecomment-1801442932 > > ![image](https://user-images.githubusercontent.com/32387433/281345992-ac951796-1659-4716-8fa4-d661a2a47d0e.png) Fine for me too. > > It work well in CI phase https://githu

Re: [I] [Bug] jdbc to jdbc数据同步任务卡住&数据量与日志打印不符(2.3.2版本正常) [seatunnel]

2023-11-08 Thread via GitHub
happyboy1024 commented on issue #5806: URL: https://github.com/apache/seatunnel/issues/5806#issuecomment-1801435996 > [seatunnel-engine-server.log](https://github.com/apache/seatunnel/files/13292824/seatunnel-engine-server.log) I used the dolphinscheduler 3.1.8 and seatunnel 2.3.3 env

Re: [PR] [Feature] add dockerfile [seatunnel]

2023-11-08 Thread via GitHub
liunaijie commented on PR #5526: URL: https://github.com/apache/seatunnel/pull/5526#issuecomment-1801431797 > ![image](https://user-images.githubusercontent.com/32387433/281345992-ac951796-1659-4716-8fa4-d661a2a47d0e.png) Fine for me too. It work well in CI phase https://github.com/

Re: [PR] [Hotfix][SQL Transform] Fix cast to timestamp, date, time bug [seatunnel]

2023-11-08 Thread via GitHub
EricJoy2048 commented on PR #5812: URL: https://github.com/apache/seatunnel/pull/5812#issuecomment-1801419439 > https://user-images.githubusercontent.com/14371345/281344554-db5546ca-da94-47a3-b992-67568855c29f.png";> fixed -- This is an automated message from the Apache Git Service.

Re: [PR] [Feature] add dockerfile [seatunnel]

2023-11-08 Thread via GitHub
Hisoka-X commented on PR #5526: URL: https://github.com/apache/seatunnel/pull/5526#issuecomment-1801418052 ![image](https://github.com/apache/seatunnel/assets/32387433/ac951796-1659-4716-8fa4-d661a2a47d0e) Fine for me too. -- This is an automated message from the Apache Git Service. To

Re: [PR] [Hotfix][SQL Transform] Fix cast to timestamp, date, time bug [seatunnel]

2023-11-08 Thread via GitHub
hailin0 commented on PR #5812: URL: https://github.com/apache/seatunnel/pull/5812#issuecomment-1801412634 https://github.com/apache/seatunnel/assets/14371345/db5546ca-da94-47a3-b992-67568855c29f";> -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] [Feature] add dockerfile [seatunnel]

2023-11-08 Thread via GitHub
liunaijie commented on PR #5526: URL: https://github.com/apache/seatunnel/pull/5526#issuecomment-1801397903 > > One question: when i pakcage locally, i find the `starter` jar has version info. like `seatunnel-starter-2.3.3-SNAPSHOT.jar` not the format `seatunnel-starter.jar`. so when i subm

Re: [PR] [Feature] add dockerfile [seatunnel]

2023-11-08 Thread via GitHub
Hisoka-X commented on PR #5526: URL: https://github.com/apache/seatunnel/pull/5526#issuecomment-1801349912 > One question: when i pakcage locally, i find the `starter` jar has version info. like `seatunnel-starter-2.3.3-SNAPSHOT.jar` not the format `seatunnel-starter.jar`. so when i submit

[PR] [Improve][MongoDB] Implement TableSourceFactory to create mongodb source [seatunnel]

2023-11-08 Thread via GitHub
hailin0 opened a new pull request, #5813: URL: https://github.com/apache/seatunnel/pull/5813 ### Purpose of this pull request [MongoDB] Implement TableSourceFactory to create mongodb source ### Does this PR introduce _any_ user-facing change? No ### How was this pa

Re: [PR] [Hotfix][SQL Transform] Fix cast to timestamp, date, time bug [seatunnel]

2023-11-08 Thread via GitHub
hailin0 commented on PR #5812: URL: https://github.com/apache/seatunnel/pull/5812#issuecomment-1801274029 LGTM waiting for ci passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp