[GitHub] [incubator-seatunnel] kyehe opened a new issue, #4367: [Bug] [Connector-V2][Hive] NullPointerException or IndexOutOfBoundsException when addRowBatch() exected.
kyehe opened a new issue, #4367:
URL: https://github.com/apache/incubator-seatunnel/issues/4367
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22)
and found no similar issues.
### What happened
source: mysql (splitted by primary key 'id' into 10)
sink: hive
spark: 2 task runs in parallel
NullPointerException and IndexOutOfBoundsException occured when receive some
row to write. the detail of stack was below.
I picked thess exception data in exception log stack to sync with a new job,
it can be sinked into hive without any exception.
### SeaTunnel Version
2.3.0
### SeaTunnel Config
```conf
env {
spark.app.name = " SeaTunnel Spark Job"
spark.dynamicAllocation.enabled = false
spark.executor.instances = 2
spark.executor.cores = 1
spark.executor.memory = "10g"
spark.driver.memory = "1g"
spark.dynamicAllocation.minExecutors = 1
spark.executor.memoryOverhead = 1g
spark.executor.heartbeatInterval = 120s
parallelism = 10
}
source {
jdbc {
driver = "com.mysql.cj.jdbc.Driver"
table = ["xxx"]
url =
"jdbc:mysql://?useUnicode=true&characterEncoding=utf8&useSSL=false&useFastDateParsing=false&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true&autoReconnect=true"
user = "xxx"
password = "xxx"
query = "select xxx from xxx"
result_table_name = "source_table"
partition_column = "id"
partition_num = 10
}
}
transform {
}
sink {
# choose stdout output plugin to output data to console
Hive {
source_table_name = "source_table"
metastore_uri = "xxx"
table_name = "xxx.xxx"
partition_dir_expression = "dt=2023-03-16"
tmp_path = "hdfs://xxx/tmp/seatunnel"
defaultFS = "hdfs://xxx"
password = "xxx"
username = "xxx"
}
}
```
### Running Command
```shell
spark-submit \
--class org.apache.seatunnel.core.starter.spark.SeatunnelSpark \
--jars
connector-hive-2.1.3-SNAPSHOT.jar,connector-jdbc-2.1.3-SNAPSHOT.jar \
--files "spark.batch.mysql2hive_file.conf" \
seatunnel-spark-starter.jar \
-m "master" \
-e "cluster" \
--config spark.batch.mysql2hive_file.conf
```
### Error Exception
```log
NullPointerException Stack:
23/03/17 14:45:13 INFO source.JdbcSourceSplitEnumerator: Assigning splits to
readers [JdbcSourceSplit(parameterValues=[29678096, 33917822], splitId=7)]
23/03/17 14:45:15 INFO impl.PhysicalFsWriter: ORC writer created for path:
hdfs://xxx/hive/tmp/seatunnel/166c8e02987647b2b355717fee1c7ddc/T_166c8e02987647b2b355717fee1c7ddc_7_1/dt=2023-03-16/T_166c8e02987647b2b355717fee1c7ddc_7_1.orc_0.orc
with stripeSize: 67108864 blockSize: 268435456 compression: SNAPPY bufferSize:
262144
23/03/17 14:45:15 INFO impl.WriterImpl: ORC writer created for path:
hdfs://xxx/hive/tmp/seatunnel/166c8e02987647b2b355717fee1c7ddc/T_166c8e02987647b2b355717fee1c7ddc_7_1/dt=2023-03-16/T_166c8e02987647b2b355717fee1c7ddc_7_1.orc_0.orc
with stripeSize: 67108864 blockSize: 268435456 compression: SNAPPY bufferSize:
262144
23/03/17 14:50:23 ERROR util.Utils: Aborting task
java.lang.RuntimeException: Write data error, please check
at
org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:65)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:32)
at
org.apache.seatunnel.translation.spark.sink.SparkDataWriter.write(SparkDataWriter.java:58)
at
org.apache.seatunnel.translation.spark.sink.SparkDataWriter.write(SparkDataWriter.java:37)
at
org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$$anonfun$run$3.apply(WriteToDataSourceV2Exec.scala:118)
at
org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$$anonfun$run$3.apply(WriteToDataSourceV2Exec.scala:116)
at
org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1397)
at
org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:146)
at
org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:67)
at
org.apache.spark.sql.execution.datasources.v2.WriteToDataSourceV2Exec$$anonfun$doExecute$2.apply(WriteToDataSourceV2Exec.scala:66)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:121)
at
org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tr
[GitHub] [incubator-seatunnel] zorrofox commented on issue #4162: [Bug] [Connect jdbc] ErrorDescription:[No suitable dialect factory found]
zorrofox commented on issue #4162: URL: https://github.com/apache/incubator-seatunnel/issues/4162#issuecomment-1473457444 I have the same problem in Java 11 in docker `eclipse-temurin:11-jre-jammy`, and I will try `eclipse-temurin:8-jre-jammy`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] kyehe commented on issue #4367: [Bug] [Connector-V2][Hive] NullPointerException or IndexOutOfBoundsException when addRowBatch() executed.
kyehe commented on issue #4367: URL: https://github.com/apache/incubator-seatunnel/issues/4367#issuecomment-1473458039 table schema: CREATE TABLE `xxx` ( `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增主键', `file_id` bigint(20) unsigned NOT NULL COMMENT '', `file_version` bigint(20) NOT NULL COMMENT '', `file_content` longtext COMMENT '文件内容', `commit_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '', `commit_user` varchar(250) NOT NULL COMMENT '', `comment` text COMMENT '', `schedule_content` longtext COMMENT '', `schedule_uuid` varchar(512) NOT NULL DEFAULT '' COMMENT '', `file_type` int(11) NOT NULL COMMENT '', `use_type` int(11) NOT NULL COMMENT '', `file_desc` varchar(250) NOT NULL DEFAULT '' COMMENT '', `file_name` varchar(512) NOT NULL DEFAULT '' COMMENT '', `status` int(11) NOT NULL COMMENT '', `change_type` int(11) NOT NULL COMMENT '', `is_current_prod` int(11) NOT NULL DEFAULT '0' COMMENT '', `project_id` bigint(20) unsigned NOT NULL COMMENT '项目id', `file_property_content` longtext COMMENT '', `parent_file_id` bigint(20) NOT NULL DEFAULT '0' COMMENT ', `wrap_content` longtext COMMENT '', PRIMARY KEY (`id`), UNIQUE KEY `idx_fid_fversion` (`file_id`,`file_version`), KEY `idx_uuid_version` (`file_version`,`schedule_uuid`), KEY `idx_commituser_status` (`commit_user`,`status`), KEY `idx_project_id` (`project_id`), KEY `idx_fid_currentprod` (`file_id`,`is_current_prod`) USING BTREE, KEY `idx_schedule_uuid_is_current_prod_change_type` (`schedule_uuid`,`is_current_prod`,`change_type`), KEY `idx_is_current_prod` (`is_current_prod`) ) ENGINE=InnoDB AUTO_INCREMENT=42400405 DEFAULT CHARSET=utf8mb4 COMMENT='文件版本表' Hive CREATE TABLE `xxx`( `id` string COMMENT '自增主键', `file_id` string COMMENT '', `file_version` bigint COMMENT '', `file_content` string COMMENT '', `commit_time` timestamp COMMENT '', `commit_user` string COMMENT '', `comment` string COMMENT '', `schedule_content` string COMMENT '', `schedule_uuid` string COMMENT '', `file_type` bigint COMMENT '', `use_type` bigint COMMENT '', `file_desc` string COMMENT '', `file_name` string COMMENT '', `status` bigint COMMENT '', `change_type` bigint COMMENT '', `is_current_prod` bigint COMMENT '', `project_id` string COMMENT '项目id', `file_property_content` string COMMENT '', `parent_file_id` bigint COMMENT '', `wrap_content` string COMMENT '') COMMENT 'xxx' PARTITIONED BY ( `dt` string COMMENT '') ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] MonsterChenzhuo commented on pull request #4195: [Hotfix][Format][Canal-Json] Fix json deserialize NPE
MonsterChenzhuo commented on PR #4195: URL: https://github.com/apache/incubator-seatunnel/pull/4195#issuecomment-1473563586 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] feng01world commented on issue #3929: [Bug] [seatunnel-connectors-v2] when I use connector-cdc-mysql to read mysql-bin data,a ClassCastException error is reported
feng01world commented on issue #3929: URL: https://github.com/apache/incubator-seatunnel/issues/3929#issuecomment-1473568349  ![Uploading image.png…]() I have done it successfully with 2.31-SNAPTSHOT. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] feng01world commented on issue #3929: [Bug] [seatunnel-connectors-v2] when I use connector-cdc-mysql to read mysql-bin data,a ClassCastException error is reported
feng01world commented on issue #3929: URL: https://github.com/apache/incubator-seatunnel/issues/3929#issuecomment-1473570264  ![Uploading image.png…]() I have realized it successfully with 2.31-SNAPTSHOT. / -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer merged pull request #4195: [Hotfix][Format][Canal-Json] Fix json deserialize NPE
TyrantLucifer merged PR #4195: URL: https://github.com/apache/incubator-seatunnel/pull/4195 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[incubator-seatunnel] branch dev updated: [Hotfix][Format][Canal-Json] Fix json deserialize NPE (#4195)
This is an automated email from the ASF dual-hosted git repository.
tyrantlucifer pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/incubator-seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new 421baf5bb [Hotfix][Format][Canal-Json] Fix json deserialize NPE (#4195)
421baf5bb is described below
commit 421baf5bb0292e26970e3a27b24744ec631f1164
Author: zhilinli <76689593+zhilinli...@users.noreply.github.com>
AuthorDate: Fri Mar 17 18:03:51 2023 +0800
[Hotfix][Format][Canal-Json] Fix json deserialize NPE (#4195)
* [Bug][seatunnel-format-json][canal-json] Fix JSON deserialize data Null
pointer prevention
* [Bug][seatunnel-format-json][canal-json] Fix JSON deserialize data Null
pointer prevention
* [Bug][seatunnel-format-json][canal-json] Fix JSON deserialize data Null
pointer prevention
* [Hotfix][Format][Canal-Json] Fix json deserialize NPE
* [Hotfix][Format][Canal-Json] Fix json deserialize NPE #4195
* [Hotfix][Format][Canal-Json] Fix json deserialize NPE #4195
* [Hotfix][Format][Canal-Json] Fix json deserialize NPE #4195
* [Hotfix][Format][Canal-Json] Fix json deserialize NPE #4195
* [Hotfix][Format][Canal-Json] Fix json deserialize NPE #4195
* [Hotfix][Format][Canal-Json] Fix json deserialize NPE #4195
* [Hotfix][Format][Canal-Json] Fix json deserialize NPE #4195
---
.../json/canal/CanalJsonDeserializationSchema.java | 33 ++
1 file changed, 21 insertions(+), 12 deletions(-)
diff --git
a/seatunnel-formats/seatunnel-format-json/src/main/java/org/apache/seatunnel/format/json/canal/CanalJsonDeserializationSchema.java
b/seatunnel-formats/seatunnel-format-json/src/main/java/org/apache/seatunnel/format/json/canal/CanalJsonDeserializationSchema.java
index 2c478f619..948c0e91e 100644
---
a/seatunnel-formats/seatunnel-format-json/src/main/java/org/apache/seatunnel/format/json/canal/CanalJsonDeserializationSchema.java
+++
b/seatunnel-formats/seatunnel-format-json/src/main/java/org/apache/seatunnel/format/json/canal/CanalJsonDeserializationSchema.java
@@ -59,6 +59,10 @@ public class CanalJsonDeserializationSchema implements
DeserializationSchema out) {
if (message == null) {
return;
}
ObjectNode jsonNode = (ObjectNode) convertBytes(message);
assert jsonNode != null;
-if (database != null) {
-if
(!databasePattern.matcher(jsonNode.get(FIELD_DATABASE).asText()).matches()) {
-return;
-}
+if (database != null
+&&
!databasePattern.matcher(jsonNode.get(FIELD_DATABASE).asText()).matches()) {
+return;
}
-if (table != null) {
-if
(!tablePattern.matcher(jsonNode.get(FIELD_TABLE).asText()).matches()) {
-return;
-}
+if (table != null &&
!tablePattern.matcher(jsonNode.get(FIELD_TABLE).asText()).matches()) {
+return;
}
JsonNode dataNode = jsonNode.get(FIELD_DATA);
String type = jsonNode.get(FIELD_TYPE).asText();
+// When a null value is encountered, an exception needs to be thrown
for easy sensing
+if (dataNode == null || dataNode.isNull()) {
+// We'll skip the query or create or alter event data
+if (OP_QUERY.equals(type) || OP_CREATE.equals(type) ||
OP_ALTER.equals(type)) {
+return;
+}
+throw new SeaTunnelJsonFormatException(
+CommonErrorCode.JSON_OPERATION_FAILED,
+format("Null data value \"%s\" Cannot send downstream",
new String(message)));
+}
if (OP_INSERT.equals(type)) {
for (int i = 0; i < dataNode.size(); i++) {
SeaTunnelRow row = convertJsonNode(dataNode.get(i));
@@ -161,9 +173,6 @@ public class CanalJsonDeserializationSchema implements
DeserializationSchema
[GitHub] [incubator-seatunnel] TyrantLucifer commented on pull request #4364: [Feature][Connector-V2][Kafka]Kafka source supports data deserializat…
TyrantLucifer commented on PR #4364: URL: https://github.com/apache/incubator-seatunnel/pull/4364#issuecomment-1473602451 Add e2e test case to verify this pull reqeust. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] zorrofox commented on issue #4162: [Bug] [Connect jdbc] ErrorDescription:[No suitable dialect factory found]
zorrofox commented on issue #4162: URL: https://github.com/apache/incubator-seatunnel/issues/4162#issuecomment-1473613641 I think this indeed the Java11 issue, can someone look at this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] FWLamb commented on a diff in pull request #4084: [HotFix][Transform-v2][Spark] fix FieldMapper ClassCastException
FWLamb commented on code in PR #4084: URL: https://github.com/apache/incubator-seatunnel/pull/4084#discussion_r1140052518 ## seatunnel-core/seatunnel-spark-starter/seatunnel-spark-2-starter/src/main/java/org/apache/seatunnel/core/starter/spark/execution/TransformExecuteProcessor.java: ## @@ -125,6 +130,22 @@ private Dataset sparkTransform(SeaTunnelTransform transform, Dataset s transform.setTypeInfo(seaTunnelDataType); StructType structType = (StructType) TypeConverterUtils.convert(transform.getProducedType()); +StructField[] newStructFields = Review Comment: Modifications in TypeConverterUtils.convert may affect other modules. The root cause of this problem is that for `TIMESTAMP` Type, the `TimestampType` in spark is returned in `TypeConverterUtils.convert`, while the `Long` Type is returned in `InternalRowConverter.convert`. Inconsistent types cause spark to report an error. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] 521daichen commented on pull request #4364: [Feature][Connector-V2][Kafka]Kafka source supports data deserializat…
521daichen commented on PR #4364: URL: https://github.com/apache/incubator-seatunnel/pull/4364#issuecomment-1473690432 > Add e2e test case to verify this pull reqeust. Thanks The function configuration has been added to the kafka-e2e project, and the local docker verification has been completed. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on pull request #4364: [Feature][Connector-V2][Kafka]Kafka source supports data deserializat…
TyrantLucifer commented on PR #4364: URL: https://github.com/apache/incubator-seatunnel/pull/4364#issuecomment-1473752123 > > Add e2e test case to verify this pull reqeust. Thanks > > The function configuration has been added to the kafka-e2e project, and the local docker verification has been completed. > > Thanks. BTW, SeaTunnel use the spotless plugin to guarantee the style of code, so before you submit pull reqeuest you should execute `mvn spotless:apply` to format you code -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] 521daichen commented on pull request #4364: [Feature][Connector-V2][Kafka]Kafka source supports data deserializat…
521daichen commented on PR #4364: URL: https://github.com/apache/incubator-seatunnel/pull/4364#issuecomment-1473950818 > Add e2e test case to verify this pull reqeust. Thanks > Thanks a lot for the reminder. I execute locally mvn spotless: check mvn spotless:apply returns successfully, but it is not clear why CI did not pass [INFO] SeaTunnel : E2E : Connector V2 : DataHub ... SUCCESS [ 0.015 s] [INFO] SeaTunnel : E2E : Connector V2 : Mongodb ... SUCCESS [ 0.015 s] [INFO] SeaTunnel : E2E : Connector V2 : Hbase . SUCCESS [ 0.014 s] [INFO] connector-maxcompute-e2e ... SUCCESS [ 0.014 s] [INFO] SeaTunnel : E2E : Engine : . SUCCESS [ 0.013 s] [INFO] SeaTunnel : E2E : Engine : Base SUCCESS [ 0.127 s] [INFO] SeaTunnel : E2E : Engine : Console . SUCCESS [ 0.016 s] [INFO] SeaTunnel : E2E : Transforms V2 SUCCESS [ 0.015 s] [INFO] SeaTunnel : E2E : Transforms V2 : Common ... SUCCESS [ 0.015 s] [INFO] SeaTunnel : E2E : Transforms V2 : Part 1 ... SUCCESS [ 0.015 s] [INFO] SeaTunnel : E2E : Transforms V2 : Part 2 ... SUCCESS [ 0.016 s] [INFO] SeaTunnel : Dist ... SUCCESS [ 0.013 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 8.417 s [INFO] Finished at: 2023-03-17T22:41:47+08:00 [INFO] ➜ incubator-seatunnel git:(521daichen/kafka-source-support-error-handle) mvn spotless:apply -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] zorrofox commented on issue #4340: How to set multiple sources at same time in one task
zorrofox commented on issue #4340:
URL:
https://github.com/apache/incubator-seatunnel/issues/4340#issuecomment-1473997707
I think your sink part should be:
```
sink {
clickhouse {
host = "1"
database = "xx"
table = "sink_table"
fields = []
username = "default"
password = "default"
bulk_size = 400
clickhouse.socket_timeout = 30
clickhouse.connection_timeout = 30
clickhouse.dataTransferTimeout = 30
retry = 3
source_table_name= result_table_A
}
clickhouse {
host = "1"
database = "xx"
table = "sink_table"
fields = []
username = "default"
password = "default"
bulk_size = 400
clickhouse.socket_timeout = 30
clickhouse.connection_timeout = 30
clickhouse.dataTransferTimeout = 30
retry = 3
source_table_name= result_table_B
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer opened a new pull request, #4368: [Hotfix][Docs] Fix docs code style
TyrantLucifer opened a new pull request, #4368: URL: https://github.com/apache/incubator-seatunnel/pull/4368 ## Purpose of this pull request ## Check list * [ ] Code changed are covered with tests, or it does not need tests for reason: * [ ] If any new Jar binary package adding in your PR, please add License Notice according [New License Guide](https://github.com/apache/incubator-seatunnel/blob/dev/docs/en/contribution/new-license.md) * [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs * [ ] If you are contributing the connector code, please check that the following files are updated: 1. Update change log that in connector document. For more details you can refer to [connector-v2](https://github.com/apache/incubator-seatunnel/tree/dev/docs/en/connector-v2) 2. Update [plugin-mapping.properties](https://github.com/apache/incubator-seatunnel/blob/dev/plugin-mapping.properties) and add new connector information in it 3. Update the pom file of [seatunnel-dist](https://github.com/apache/incubator-seatunnel/blob/dev/seatunnel-dist/pom.xml) * [ ] Update the [`release-note`](https://github.com/apache/incubator-seatunnel/blob/dev/release-note.md). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #4137: [Feature][Connector-V2] connector-kafka source support data conversion extracted by kafka connect source
github-actions[bot] commented on issue #4137: URL: https://github.com/apache/incubator-seatunnel/issues/4137#issuecomment-1474539790 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #4131: [Feature][Connector-V2][ES]Support DSL
github-actions[bot] commented on issue #4131: URL: https://github.com/apache/incubator-seatunnel/issues/4131#issuecomment-1474539798 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] closed issue #4081: [Bug][Transform-v2][Spark] FieldMapper ClassCastException
github-actions[bot] closed issue #4081: [Bug][Transform-v2][Spark] FieldMapper ClassCastException URL: https://github.com/apache/incubator-seatunnel/issues/4081 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #4081: [Bug][Transform-v2][Spark] FieldMapper ClassCastException
github-actions[bot] commented on issue #4081: URL: https://github.com/apache/incubator-seatunnel/issues/4081#issuecomment-1474539817 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #4079: [Bug] [common] fix possible unrecognized NestedField name
github-actions[bot] commented on issue #4079: URL: https://github.com/apache/incubator-seatunnel/issues/4079#issuecomment-1474539827 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] closed issue #4079: [Bug] [common] fix possible unrecognized NestedField name
github-actions[bot] closed issue #4079: [Bug] [common] fix possible unrecognized NestedField name URL: https://github.com/apache/incubator-seatunnel/issues/4079 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #4077: [Feature][Zeta] Add agent to solve cross-cluster and multi-language source
github-actions[bot] commented on issue #4077: URL: https://github.com/apache/incubator-seatunnel/issues/4077#issuecomment-1474539841 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] closed issue #4077: [Feature][Zeta] Add agent to solve cross-cluster and multi-language source
github-actions[bot] closed issue #4077: [Feature][Zeta] Add agent to solve cross-cluster and multi-language source URL: https://github.com/apache/incubator-seatunnel/issues/4077 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #4066: [Feature][CDC][Pulsar] Support read canal format message
github-actions[bot] commented on issue #4066: URL: https://github.com/apache/incubator-seatunnel/issues/4066#issuecomment-1474539862 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] closed issue #4066: [Feature][CDC][Pulsar] Support read canal format message
github-actions[bot] closed issue #4066: [Feature][CDC][Pulsar] Support read canal format message URL: https://github.com/apache/incubator-seatunnel/issues/4066 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] closed issue #4055: [Feature][Flink] Make Flink engine support multi-table
github-actions[bot] closed issue #4055: [Feature][Flink] Make Flink engine support multi-table URL: https://github.com/apache/incubator-seatunnel/issues/4055 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #4055: [Feature][Flink] Make Flink engine support multi-table
github-actions[bot] commented on issue #4055: URL: https://github.com/apache/incubator-seatunnel/issues/4055#issuecomment-1474539888 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] dijiekstra merged pull request #4368: [Hotfix][Docs] Fix docs code style
dijiekstra merged PR #4368: URL: https://github.com/apache/incubator-seatunnel/pull/4368 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[incubator-seatunnel] branch dev updated: [Hotfix][Docs] Fix docs code style (#4368)
This is an automated email from the ASF dual-hosted git repository. dijiekstra pushed a commit to branch dev in repository https://gitbox.apache.org/repos/asf/incubator-seatunnel.git The following commit(s) were added to refs/heads/dev by this push: new 56fbc8ed2 [Hotfix][Docs] Fix docs code style (#4368) 56fbc8ed2 is described below commit 56fbc8ed272acb17434f44294f3c98617a5ab4a6 Author: Tyrantlucifer AuthorDate: Sat Mar 18 08:55:35 2023 +0800 [Hotfix][Docs] Fix docs code style (#4368) --- docs/en/connector-v2/sink/SelectDB-Cloud.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/en/connector-v2/sink/SelectDB-Cloud.md b/docs/en/connector-v2/sink/SelectDB-Cloud.md index 4970dc4d0..5b182327a 100644 --- a/docs/en/connector-v2/sink/SelectDB-Cloud.md +++ b/docs/en/connector-v2/sink/SelectDB-Cloud.md @@ -22,7 +22,7 @@ Version Supported ## Options -| name | type | required | default value | +|name| type | required | default value | |||--|| | load-url | string | yes | - | | jdbc-url | string | yes | - |
[GitHub] [incubator-seatunnel] liugddx commented on issue #4143: [Umbrella] [Zeta] [REST-API] Provide rest api for zeta
liugddx commented on issue #4143: URL: https://github.com/apache/incubator-seatunnel/issues/4143#issuecomment-1474709568 > I am happy to help. What's the progress? Let me know if you need anything. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on issue #4143: [Umbrella] [Zeta] [REST-API] Provide rest api for zeta
liugddx commented on issue #4143: URL: https://github.com/apache/incubator-seatunnel/issues/4143#issuecomment-1474710054 > maybe i want to try. What's the progress? Let me know if you need anything. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on issue #4145: [Feature][Zeta][REST-API] Get all completed jobs information.
liugddx commented on issue #4145: URL: https://github.com/apache/incubator-seatunnel/issues/4145#issuecomment-1474713315 > maybe i want to try. What's the progress? Let me know if you need anything. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #4315: [Feature][Zeta][REST-API]Get system monitoring information
liugddx commented on PR #4315: URL: https://github.com/apache/incubator-seatunnel/pull/4315#issuecomment-1474715247 Please add e2e case like #4140 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@seatunnel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
