Re: [I] [Bug] [Mysql-cdc] Mysql-cdc cannot automatically synchronize newly created fields in DDL [seatunnel]

2024-06-03 Thread via GitHub
ixiaoyi93 commented on issue #6450: URL: https://github.com/apache/seatunnel/issues/6450#issuecomment-2146746364 > > I'm having the same problem. Using mysql cdc to synchronize data to Doris, after adding new columns in mysql, it doesn't give synchronization to Doris. > > I have used

Re: [I] [Bug] [Mysql-cdc] Mysql-cdc cannot automatically synchronize newly created fields in DDL [seatunnel]

2024-06-03 Thread via GitHub
czxin788 commented on issue #6450: URL: https://github.com/apache/seatunnel/issues/6450#issuecomment-2146741181 > I'm having the same problem. Using mysql cdc to synchronize data to Doris, after adding new columns in mysql, it doesn't give synchronization to Doris. I have used otter o

Re: [I] [Bug] [Mysql-cdc] Mysql-cdc cannot automatically synchronize newly created fields in DDL [seatunnel]

2024-06-03 Thread via GitHub
ixiaoyi93 commented on issue #6450: URL: https://github.com/apache/seatunnel/issues/6450#issuecomment-2146733033 I'm having the same problem. Using mysql cdc to synchronize data to Doris, after adding new columns in mysql, it doesn't give synchronization to Doris. -- This is an automated

Re: [I] [Bug] [connector-kudu] Wrong kerberos config for kudu connector [seatunnel]

2024-06-03 Thread via GitHub
JackeyLee007 closed issue #6903: [Bug] [connector-kudu] Wrong kerberos config for kudu connector URL: https://github.com/apache/seatunnel/issues/6903 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] [Improve][Connector-V2] Support schema evolution for mysql-cdc [seatunnel]

2024-06-03 Thread via GitHub
dailai commented on code in PR #6929: URL: https://github.com/apache/seatunnel/pull/6929#discussion_r1625253246 ## seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql/src/main/java/org/apache/seatunnel/connectors/seatunnel/cdc/mysql/config/MySqlSourceConfigFactory.java: ##

Re: [PR] [Improve][Connector-V2] Support schema evolution for mysql-cdc [seatunnel]

2024-06-03 Thread via GitHub
dailai commented on code in PR #6929: URL: https://github.com/apache/seatunnel/pull/6929#discussion_r1625253101 ## seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql/src/main/java/org/apache/seatunnel/connectors/seatunnel/cdc/mysql/source/parser/CustomMySqlAntlrDdlParser.java: ##

Re: [PR] [Improve][Connector-V2] Support schema evolution for mysql-cdc [seatunnel]

2024-06-03 Thread via GitHub
hailin0 commented on code in PR #6929: URL: https://github.com/apache/seatunnel/pull/6929#discussion_r1625251542 ## seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql/src/main/java/org/apache/seatunnel/connectors/seatunnel/cdc/mysql/source/parser/CustomMySqlAntlrDdlParser.java: #

[PR] Unstructured transmission extension based on connector-file [seatunnel]

2024-06-03 Thread via GitHub
gitjxm opened a new pull request, #6940: URL: https://github.com/apache/seatunnel/pull/6940 I don't know if this makes sense, but can consider it # Count counts the number of files # bytes counts the size of files -- This is an automated message from the Apache Git Ser

Re: [PR] [Improve][Connector-V2] Support schema evolution for mysql-cdc [seatunnel]

2024-06-03 Thread via GitHub
dailai commented on code in PR #6929: URL: https://github.com/apache/seatunnel/pull/6929#discussion_r1625247688 ## seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql/pom.xml: ## @@ -98,6 +98,11 @@ mysql mysql-connector-java + +

Re: [PR] [Improve][Connector-V2] Support schema evolution for mysql-cdc [seatunnel]

2024-06-03 Thread via GitHub
dailai commented on code in PR #6929: URL: https://github.com/apache/seatunnel/pull/6929#discussion_r1625246762 ## seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql/pom.xml: ## @@ -98,6 +98,11 @@ mysql mysql-connector-java + +

Re: [PR] [Improve][Connector-V2] Support schema evolution for mysql-cdc [seatunnel]

2024-06-03 Thread via GitHub
hailin0 commented on code in PR #6929: URL: https://github.com/apache/seatunnel/pull/6929#discussion_r1625245315 ## seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql/pom.xml: ## @@ -98,6 +98,11 @@ mysql mysql-connector-java + +

Re: [PR] Unstructured transmission extension based on connector-file [seatunnel]

2024-06-03 Thread via GitHub
gitjxm closed pull request #6939: Unstructured transmission extension based on connector-file URL: https://github.com/apache/seatunnel/pull/6939 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[PR] Unstructured transmission extension based on connector-file [seatunnel]

2024-06-03 Thread via GitHub
gitjxm opened a new pull request, #6939: URL: https://github.com/apache/seatunnel/pull/6939 I don't know if this makes sense, but can consider it # count counts the number of files # bytes counts the size of files -- This is an automated message from the Apache Git Service. T

Re: [PR] [Improve][Connector-V2] Support schema evolution for mysql-cdc [seatunnel]

2024-06-03 Thread via GitHub
dailai commented on code in PR #6929: URL: https://github.com/apache/seatunnel/pull/6929#discussion_r1625237767 ## seatunnel-connectors-v2/connector-cdc/connector-cdc-mysql/pom.xml: ## @@ -98,6 +98,11 @@ mysql mysql-connector-java + +

Re: [PR] [Improve][Connector-V2] Support schema evolution for mysql-cdc [seatunnel]

2024-06-03 Thread via GitHub
hailin0 commented on code in PR #6929: URL: https://github.com/apache/seatunnel/pull/6929#discussion_r1625234805 ## seatunnel-connectors-v2/connector-cdc/connector-cdc-base/src/main/java/org/apache/seatunnel/connectors/cdc/base/schema/AbstractSchemaChangeResolver.java: ## @@ -0,

(seatunnel) branch dev updated: [E2E][HBase]Refactor hbase e2e (#6859)

2024-06-03 Thread wanghailin
This is an automated email from the ASF dual-hosted git repository. wanghailin pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new 1da9bd6ce4 [E2E][HBase]Refactor hbase e2e (#685

Re: [PR] [E2E][HBase]Refactor hbase e2e [seatunnel]

2024-06-03 Thread via GitHub
hailin0 merged PR #6859: URL: https://github.com/apache/seatunnel/pull/6859 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunne

Re: [I] [Feature][InfluxDB] source [seatunnel]

2024-06-03 Thread via GitHub
github-actions[bot] commented on issue #6751: URL: https://github.com/apache/seatunnel/issues/6751#issuecomment-2146343556 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is a

Re: [I] [Feature][InfluxDB] source [seatunnel]

2024-06-03 Thread via GitHub
github-actions[bot] closed issue #6751: [Feature][InfluxDB] source URL: https://github.com/apache/seatunnel/issues/6751 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[PR] [Improve] Clean checkpoint file when job FINISHED/CANCELED [seatunnel]

2024-06-03 Thread via GitHub
Hisoka-X opened a new pull request, #6938: URL: https://github.com/apache/seatunnel/pull/6938 ### Purpose of this pull request This PR try to clean checkpoint file when job FINISHED/CANCELED. Because when job FINISHED/CANCELED, the checkpoint file will be useless. #

[I] [Bug] [Cluster deploy] java.lang.NoSuchMethodError [seatunnel]

2024-06-03 Thread via GitHub
HSLife1991 opened a new issue, #6937: URL: https://github.com/apache/seatunnel/issues/6937 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happene

Re: [I] SeaTunnel community meeting Topic collect [seatunnel]

2024-06-03 Thread via GitHub
hailin0 commented on issue #6936: URL: https://github.com/apache/seatunnel/issues/6936#issuecomment-2144986380 @TaoZex https://github.com/apache/seatunnel/pull/6348 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [I] SeaTunnel community meeting Topic collect [seatunnel]

2024-06-03 Thread via GitHub
hailin0 commented on issue #6936: URL: https://github.com/apache/seatunnel/issues/6936#issuecomment-2144985600 @EricJoy2048 https://github.com/apache/seatunnel/pull/6885 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] SeaTunnel community meeting Topic collect [seatunnel]

2024-06-03 Thread via GitHub
hailin0 commented on issue #6936: URL: https://github.com/apache/seatunnel/issues/6936#issuecomment-2144984471 @dailai https://github.com/apache/seatunnel/pull/6740/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] SeaTunnel community meeting Topic collect [seatunnel]

2024-06-03 Thread via GitHub
hailin0 commented on issue #6936: URL: https://github.com/apache/seatunnel/issues/6936#issuecomment-2144978978 @hailin0 https://github.com/apache/seatunnel/pull/6864 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] SeaTunnel community meeting Topic collect [seatunnel]

2024-06-03 Thread via GitHub
hailin0 commented on issue #6936: URL: https://github.com/apache/seatunnel/issues/6936#issuecomment-2144976876 @Hisoka-X https://github.com/apache/seatunnel/pull/6826 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[I] SeaTunnel community meeting Topic collect [seatunnel]

2024-06-03 Thread via GitHub
hailin0 opened a new issue, #6936: URL: https://github.com/apache/seatunnel/issues/6936 ### Search before asking - [X] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement. #

(seatunnel) branch dev updated: [Feature][RestAPI] Add overview api (#6883)

2024-06-03 Thread wanghailin
This is an automated email from the ASF dual-hosted git repository. wanghailin pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new 2cc82bd4dc [Feature][RestAPI] Add overview api

Re: [PR] [Feature][RestAPI] Add overview api [seatunnel]

2024-06-03 Thread via GitHub
hailin0 merged PR #6883: URL: https://github.com/apache/seatunnel/pull/6883 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunne

Re: [I] [Bug] Can not find catalog table with factoryId [Postgres] [seatunnel]

2024-06-03 Thread via GitHub
chess3cake commented on issue #6550: URL: https://github.com/apache/seatunnel/issues/6550#issuecomment-2144845145 > @Asura7969 hi,how did you resolve it? I also had the same error @Carl-Zhou-CN my config like this `env { # You can set engine configuration here execution.paral

[I] [Bug] [Hive Connector] Hive Connector Kerberos auto Failed [seatunnel]

2024-06-03 Thread via GitHub
EricJoy2048 opened a new issue, #6935: URL: https://github.com/apache/seatunnel/issues/6935 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happen

Re: [I] [Bug] Can not find catalog table with factoryId [Postgres] [seatunnel]

2024-06-03 Thread via GitHub
chess3cake commented on issue #6550: URL: https://github.com/apache/seatunnel/issues/6550#issuecomment-2144840929 @Asura7969 hi,how did you resolve it? I also had the same error -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] [Bug] [mysql cdc] org.apache.seatunnel.connectors.seatunnel.cdc.mysql.source.MySqlDialect.getPrimaryKey< Error NullPointerException> [seatunnel]

2024-06-03 Thread via GitHub
wuchang945 commented on issue #6934: URL: https://github.com/apache/seatunnel/issues/6934#issuecomment-2144736924 Caused by: java.lang.NullPointerException at org.apache.seatunnel.connectors.seatunnel.cdc.mysql.source.MySqlDialect.getPrimaryKey(MySqlDialect.java:127) ~[connector-

Re: [PR] [Feature][Connector Hive] support hive savemode [seatunnel]

2024-06-03 Thread via GitHub
liunaijie commented on PR #6842: URL: https://github.com/apache/seatunnel/pull/6842#issuecomment-2144727147 > > ![image](https://private-user-images.githubusercontent.com/73467479/333069983-b89e6671-cfad-4c66-80dc-a67f8b659d7d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIu

Re: [PR] [Feature][Connector Hive] support hive savemode [seatunnel]

2024-06-03 Thread via GitHub
liunaijie commented on code in PR #6842: URL: https://github.com/apache/seatunnel/pull/6842#discussion_r1624112555 ## docs/en/connector-v2/sink/Hive.md: ## @@ -28,78 +33,175 @@ By default, we use 2PC commit to ensure `exactly-once` - [x] compress codec - [x] lzo -## Option

Re: [PR] [Bug][Connector-v2][KafkaSink][Problems relating to commiting transactions under EXACTLY_ONCE semantics] [seatunnel]

2024-06-03 Thread via GitHub
UFOXD commented on PR #4469: URL: https://github.com/apache/seatunnel/pull/4469#issuecomment-2144718432 when will this pr be merged ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [Feature][Connector Hive] support hive savemode [seatunnel]

2024-06-03 Thread via GitHub
liunaijie commented on code in PR #6842: URL: https://github.com/apache/seatunnel/pull/6842#discussion_r1624067504 ## docs/en/connector-v2/sink/Hive.md: ## @@ -33,7 +33,7 @@ By default, we use 2PC commit to ensure `exactly-once` | name | type | requi

Re: [PR] [Feature][Connector Hive] support hive savemode [seatunnel]

2024-06-03 Thread via GitHub
liunaijie commented on code in PR #6842: URL: https://github.com/apache/seatunnel/pull/6842#discussion_r1624053278 ## seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/catalog/HiveJDBCCatalog.java: ## @@ -0,0 +1,451 @@ +/* + * Li

[I] [Bug] [mysql cdc] org.apache.seatunnel.connectors.seatunnel.cdc.mysql.source.MySqlDialect.getPrimaryKey< Error NullPointerException> [seatunnel]

2024-06-03 Thread via GitHub
wuchang945 opened a new issue, #6934: URL: https://github.com/apache/seatunnel/issues/6934 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happene

Re: [PR] [Feature][Connector Hive] support hive savemode [seatunnel]

2024-06-03 Thread via GitHub
EricJoy2048 commented on code in PR #6842: URL: https://github.com/apache/seatunnel/pull/6842#discussion_r1624012376 ## docs/en/connector-v2/sink/Hive.md: ## @@ -33,7 +33,7 @@ By default, we use 2PC commit to ensure `exactly-once` | name | type | req

Re: [PR] [Feature][Connector Hive] support hive savemode [seatunnel]

2024-06-03 Thread via GitHub
zhilinli123 commented on code in PR #6842: URL: https://github.com/apache/seatunnel/pull/6842#discussion_r1624003182 ## seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/catalog/HiveJDBCCatalog.java: ## @@ -0,0 +1,451 @@ +/* + *

Re: [PR] [Feature][Connector Hive] support hive savemode [seatunnel]

2024-06-03 Thread via GitHub
zhilinli123 commented on code in PR #6842: URL: https://github.com/apache/seatunnel/pull/6842#discussion_r1623945313 ## docs/en/connector-v2/source/Hive.md: ## @@ -31,72 +36,62 @@ Read all the data in a split in a pollNext call. What splits are read will be sa - [x] orc -

Re: [PR] [Feature][Connector Hive] support hive savemode [seatunnel]

2024-06-03 Thread via GitHub
zhilinli123 commented on code in PR #6842: URL: https://github.com/apache/seatunnel/pull/6842#discussion_r1623944873 ## docs/en/connector-v2/source/Hive.md: ## @@ -6,12 +6,17 @@ Read data from Hive. -:::tip +## Support Versions -In order to use this connector, You must en

Re: [PR] [Feature][Connector Hive] support hive savemode [seatunnel]

2024-06-03 Thread via GitHub
zhilinli123 commented on code in PR #6842: URL: https://github.com/apache/seatunnel/pull/6842#discussion_r1623944204 ## docs/en/connector-v2/sink/Hive.md: ## @@ -28,78 +33,175 @@ By default, we use 2PC commit to ensure `exactly-once` - [x] compress codec - [x] lzo -## Opti

Re: [PR] [Feature][Connector Hive] support hive savemode [seatunnel]

2024-06-03 Thread via GitHub
zhilinli123 commented on code in PR #6842: URL: https://github.com/apache/seatunnel/pull/6842#discussion_r1623942806 ## docs/en/connector-v2/sink/Hive.md: ## @@ -6,12 +6,17 @@ Write data to Hive. -:::tip +## Support Versions -In order to use this connector, You must ensur

[I] How to set the master node in seatunnel2.3.3 cluster deployment? [seatunnel]

2024-06-03 Thread via GitHub
sandyustc opened a new issue, #6933: URL: https://github.com/apache/seatunnel/issues/6933 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened

Re: [PR] [E2E][HBase]Refactor hbase e2e [seatunnel]

2024-06-03 Thread via GitHub
TaoZex commented on code in PR #6859: URL: https://github.com/apache/seatunnel/pull/6859#discussion_r1623895004 ## seatunnel-e2e/seatunnel-connector-v2-e2e/connector-hbase-e2e/pom.xml: ## @@ -26,14 +26,23 @@ SeaTunnel : E2E : Connector V2 : Hbase - + +