[GitHub] [inlong] fuweng11 commented on pull request #8795: [INLONG-8794][Manager] Support add streamField and sinkField
fuweng11 commented on PR #8795: URL: https://github.com/apache/inlong/pull/8795#issuecomment-1691117552 > Maybe it is better to add some unit tests for it. Done. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] gosonzhang commented on a diff in pull request #8793: [INLONG-8791][TubeMQ] Tubemq-client-go lacks log level configuration API
gosonzhang commented on code in PR #8793:
URL: https://github.com/apache/inlong/pull/8793#discussion_r1303947305
##
inlong-tubemq/tubemq-client-twins/tubemq-client-go/client/consumer_impl.go:
##
@@ -77,6 +77,10 @@ func NewConsumer(config *config.Config) (Consumer, error) {
if err := config.ValidateConsumer(); err != nil {
return nil, err
}
+ log.SetLogLevel(config.Log.LogLevel)
Review Comment:
What should we do if multiple consumers set Log information and the set
values are inconsistent?
I personally think that the setting of the log service should be a separate
operation and should not be bound to the Consumer object
##
inlong-tubemq/tubemq-client-twins/tubemq-client-go/log/config.go:
##
@@ -39,3 +39,18 @@ var defaultConfig = &OutputConfig{
MaxAge: 3,
Level: "warn",
}
+
+// SetLogLevel set log level
+func SetLogLevel(level string) {
Review Comment:
Need to check whether the set value is legal and valid
##
inlong-tubemq/tubemq-client-twins/tubemq-client-go/log/config.go:
##
@@ -39,3 +39,18 @@ var defaultConfig = &OutputConfig{
MaxAge: 3,
Level: "warn",
}
+
+// SetLogLevel set log level
+func SetLogLevel(level string) {
+ defaultConfig.Level = level
+}
+
+// SetLogPath set log path
+func SetLogPath(path string) {
Review Comment:
It is necessary to check that the path allowed to be set is legal, and it
may be necessary to check whether there is permission to output the log
##
inlong-tubemq/tubemq-client-twins/tubemq-client-go/log/config.go:
##
@@ -39,3 +39,18 @@ var defaultConfig = &OutputConfig{
MaxAge: 3,
Level: "warn",
}
+
+// SetLogLevel set log level
+func SetLogLevel(level string) {
+ defaultConfig.Level = level
+}
+
+// SetLogPath set log path
+func SetLogPath(path string) {
+ defaultConfig.LogPath = path
+}
+
+// GetLogConfig get log config
+func GetLogConfig() *OutputConfig {
Review Comment:
It is not appropriate to directly expose objects for external settings. It
is best to add methods to operate, and then add validity checks to the set
values in the methods
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [inlong] smarteng closed pull request #8793: [INLONG-8791][TubeMQ] Tubemq-client-go lacks log level configuration API
smarteng closed pull request #8793: [INLONG-8791][TubeMQ] Tubemq-client-go lacks log level configuration API URL: https://github.com/apache/inlong/pull/8793 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] haifxu opened a new pull request, #8798: [INLONG-8797][Manager][Sort] Support multiple audit ids for data sync
haifxu opened a new pull request, #8798: URL: https://github.com/apache/inlong/pull/8798 ### Prepare a Pull Request - Fixes #8797 ### Motivation 1. Sort does not support audit ids sent by the Manager for data sync. ### Modifications 1. The manager adds audit IDs to send. 2. `JDBC`, `MySQL-CDC`, `Iceberg`, and `Starrocks` connectors support audit id. ### Verifying this change  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] bluewang opened a new pull request, #8801: [INLONG-8800][Dashboard] Owners search component optimization
bluewang opened a new pull request, #8801: URL: https://github.com/apache/inlong/pull/8801 ### Prepare a Pull Request *(Change the title refer to the following example)* - Title Example: [INLONG-XYZ][Component] Title of the pull request *(The following *XYZ* should be replaced by the actual [GitHub Issue](https://github.com/apache/inlong/issues) number)* - Fixes #XYZ ### Motivation *Explain here the context, and why you're making that change. What is the problem you're trying to solve?* ### Modifications Start searching all users when getting focus. ### Verifying this change  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] ZhaoNiuniu opened a new pull request, #8802: [INLONG-8799][Manager][Agent][Dataproxy] Fix the configuration related to "Opentelemetry".
ZhaoNiuniu opened a new pull request, #8802: URL: https://github.com/apache/inlong/pull/8802 ### Prepare a Pull Request - Title Example: [INLONG-8799][Manager][Agent][Dataproxy] Fix the configuration related to "Opentelemetry". - Fixes #8799 ### Modifications -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong-website] dockerzhang merged pull request #839: [INLONG-825][Doc] Add the quick start guide for collecting MySQL and writing to ClickHouse
dockerzhang merged PR #839: URL: https://github.com/apache/inlong-website/pull/839 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong-website] dockerzhang closed issue #825: [Doc][Translate] Add the quick start guide for collecting MySQL and writing to ClickHouse
dockerzhang closed issue #825: [Doc][Translate] Add the quick start guide for collecting MySQL and writing to ClickHouse URL: https://github.com/apache/inlong-website/issues/825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[inlong-website] branch master updated: [INLONG-825][Doc] Add the quick start guide for collecting MySQL and writing to ClickHouse (#839)
This is an automated email from the ASF dual-hosted git repository. dockerzhang pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/inlong-website.git The following commit(s) were added to refs/heads/master by this push: new c577822d46 [INLONG-825][Doc] Add the quick start guide for collecting MySQL and writing to ClickHouse (#839) c577822d46 is described below commit c577822d4686266393b6fc3570bdcc813acb88ad Author: Amelie <1433021...@qq.com> AuthorDate: Thu Aug 24 17:20:59 2023 +0800 [INLONG-825][Doc] Add the quick start guide for collecting MySQL and writing to ClickHouse (#839) Co-authored-by: joytzli Co-authored-by: Charles Zhang --- docs/quick_start/img/mysql_clickhouse/approval.png | Bin 0 -> 83618 bytes .../img/mysql_clickhouse/create_cluster_label.png | Bin 0 -> 22127 bytes .../img/mysql_clickhouse/create_data_source.png| Bin 0 -> 147936 bytes .../img/mysql_clickhouse/create_ingestion.png | Bin 0 -> 159264 bytes .../img/mysql_clickhouse/create_sink.png | Bin 0 -> 229676 bytes .../quick_start/img/mysql_clickhouse/data_page.png | Bin 0 -> 142836 bytes .../img/mysql_clickhouse/data_stream_config.png| Bin 0 -> 169165 bytes .../img/mysql_clickhouse/data_table.png| Bin 0 -> 11751 bytes docs/quick_start/img/mysql_clickhouse/datanode.png | Bin 0 -> 101757 bytes .../img/mysql_clickhouse/kafka_cluster.png | Bin 0 -> 89194 bytes docs/quick_start/img/mysql_clickhouse/result.png | Bin 0 -> 123659 bytes .../quick_start/img/mysql_clickhouse/send_data.png | Bin 0 -> 4877 bytes docs/quick_start/mysql_kafka_clickhouse_example.md | 89 + .../quick_start/img/mysql_clickhouse/approval.png | Bin 0 -> 14554 bytes .../img/mysql_clickhouse/create_cluster_label.png | Bin 0 -> 5211 bytes .../img/mysql_clickhouse/create_data_source.png| Bin 0 -> 43896 bytes .../img/mysql_clickhouse/create_ingestion.png | Bin 0 -> 42189 bytes .../img/mysql_clickhouse/create_sink.png | Bin 0 -> 108070 bytes .../quick_start/img/mysql_clickhouse/data_page.png | Bin 0 -> 72009 bytes .../img/mysql_clickhouse/data_stream_config.png| Bin 0 -> 49984 bytes .../img/mysql_clickhouse/data_table.png| Bin 0 -> 11751 bytes .../quick_start/img/mysql_clickhouse/datanode.png | Bin 0 -> 27843 bytes .../img/mysql_clickhouse/kafka_cluster.png | Bin 0 -> 10461 bytes .../quick_start/img/mysql_clickhouse/result.png| Bin 0 -> 28199 bytes .../quick_start/img/mysql_clickhouse/send_data.png | Bin 0 -> 4877 bytes .../quick_start/mysql_kafka_clickhouse_example.md | 86 26 files changed, 175 insertions(+) diff --git a/docs/quick_start/img/mysql_clickhouse/approval.png b/docs/quick_start/img/mysql_clickhouse/approval.png new file mode 100644 index 00..10071397ce Binary files /dev/null and b/docs/quick_start/img/mysql_clickhouse/approval.png differ diff --git a/docs/quick_start/img/mysql_clickhouse/create_cluster_label.png b/docs/quick_start/img/mysql_clickhouse/create_cluster_label.png new file mode 100644 index 00..8fb4d7e16f Binary files /dev/null and b/docs/quick_start/img/mysql_clickhouse/create_cluster_label.png differ diff --git a/docs/quick_start/img/mysql_clickhouse/create_data_source.png b/docs/quick_start/img/mysql_clickhouse/create_data_source.png new file mode 100644 index 00..a4a143c9a9 Binary files /dev/null and b/docs/quick_start/img/mysql_clickhouse/create_data_source.png differ diff --git a/docs/quick_start/img/mysql_clickhouse/create_ingestion.png b/docs/quick_start/img/mysql_clickhouse/create_ingestion.png new file mode 100644 index 00..8c6cbec226 Binary files /dev/null and b/docs/quick_start/img/mysql_clickhouse/create_ingestion.png differ diff --git a/docs/quick_start/img/mysql_clickhouse/create_sink.png b/docs/quick_start/img/mysql_clickhouse/create_sink.png new file mode 100644 index 00..2b0b70ebbc Binary files /dev/null and b/docs/quick_start/img/mysql_clickhouse/create_sink.png differ diff --git a/docs/quick_start/img/mysql_clickhouse/data_page.png b/docs/quick_start/img/mysql_clickhouse/data_page.png new file mode 100644 index 00..b67ba707cc Binary files /dev/null and b/docs/quick_start/img/mysql_clickhouse/data_page.png differ diff --git a/docs/quick_start/img/mysql_clickhouse/data_stream_config.png b/docs/quick_start/img/mysql_clickhouse/data_stream_config.png new file mode 100644 index 00..69d02de9e4 Binary files /dev/null and b/docs/quick_start/img/mysql_clickhouse/data_stream_config.png differ diff --git a/docs/quick_start/img/mysql_clickhouse/data_table.png b/docs/quick_start/img/mysql_clickhouse/data_table.png new file mode 100644 index 00..321406a4a2 Binary files /dev/null and b/docs/quick_start/img/mysql_clickhouse/data_table.png differ diff --git a/docs/quick_start/img/mysql_clickhouse/datanode.png b/docs/quick_start/img/mysql_clickhous
[GitHub] [inlong-website] dockerzhang closed issue #822: [Doc][Translate] Add the quick start guide for collecting file and writing to ClickHouse
dockerzhang closed issue #822: [Doc][Translate] Add the quick start guide for collecting file and writing to ClickHouse URL: https://github.com/apache/inlong-website/issues/822 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[inlong-website] branch master updated: [INLONG-822][Doc] Add the quick start guide for collecting file and writing to ClickHouse (#838)
This is an automated email from the ASF dual-hosted git repository. dockerzhang pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/inlong-website.git The following commit(s) were added to refs/heads/master by this push: new 4e60d61f30 [INLONG-822][Doc] Add the quick start guide for collecting file and writing to ClickHouse (#838) 4e60d61f30 is described below commit 4e60d61f30d4502559c555ce89616f22c28a1636 Author: Amelie <1433021...@qq.com> AuthorDate: Thu Aug 24 17:21:23 2023 +0800 [INLONG-822][Doc] Add the quick start guide for collecting file and writing to ClickHouse (#838) Co-authored-by: joytzli Co-authored-by: Charles Zhang --- docs/quick_start/file_pulsar_clickhouse_example.md | 101 + .../img/file_clickhouse/create_cluster_label.png | Bin 0 -> 22127 bytes .../img/file_clickhouse/create_pulsar_cluster.png | Bin 0 -> 36568 bytes docs/quick_start/img/file_clickhouse/datanode.png | Bin 0 -> 101757 bytes .../img/file_clickhouse/pulsar_approval.png| Bin 0 -> 109310 bytes .../img/file_clickhouse/pulsar_approve_result.png | Bin 0 -> 163376 bytes .../img/file_clickhouse/pulsar_audit.png | Bin 0 -> 161957 bytes .../img/file_clickhouse/pulsar_file_source.png | Bin 0 -> 88091 bytes .../img/file_clickhouse/pulsar_group.png | Bin 0 -> 355321 bytes .../img/file_clickhouse/pulsar_sink.png| Bin 0 -> 252822 bytes .../img/file_clickhouse/pulsar_stream.png | Bin 0 -> 192767 bytes .../img/file_clickhouse/pulsar_table.png | Bin 0 -> 47031 bytes .../quick_start/file_pulsar_clickhouse_example.md | 98 .../img/file_clickhouse/create_cluster_label.png | Bin 0 -> 5211 bytes .../img/file_clickhouse/create_pulsar_cluster.png | Bin 0 -> 30222 bytes .../quick_start/img/file_clickhouse/datanode.png | Bin 0 -> 27843 bytes .../img/file_clickhouse/pulsar_approval.png| Bin 0 -> 36519 bytes .../img/file_clickhouse/pulsar_approval_result.png | Bin 0 -> 53846 bytes .../img/file_clickhouse/pulsar_audit.png | Bin 0 -> 57854 bytes .../img/file_clickhouse/pulsar_ingestion.png | Bin 0 -> 91016 bytes .../img/file_clickhouse/pulsar_sink.png| Bin 0 -> 241208 bytes .../img/file_clickhouse/pulsar_source.png | Bin 0 -> 93424 bytes .../img/file_clickhouse/pulsar_stream.png | Bin 0 -> 79289 bytes .../img/file_clickhouse/pulsar_table.png | Bin 0 -> 47031 bytes 24 files changed, 199 insertions(+) diff --git a/docs/quick_start/file_pulsar_clickhouse_example.md b/docs/quick_start/file_pulsar_clickhouse_example.md new file mode 100644 index 00..f2a5c22022 --- /dev/null +++ b/docs/quick_start/file_pulsar_clickhouse_example.md @@ -0,0 +1,101 @@ +--- +title: File -> Pulsar -> ClickHouse Example +sidebar_position: 4 +--- + +Here we use an example to introduce how to create File -> Pulsar -> ClickHouse data ingestion. + +## Deployment +### Install InLong + +Before we begin, we need to install InLong. Here we provide two ways: +1. Install InLong with Docker by according to the [instructions here](deployment/docker.md).(Recommanded) +2. Install InLong binary according to the [instructions here](deployment/bare_metal.md). + +### Install ClickHouse +```shell +docker run -d --rm --net=host --name clickhouse -e CLICKHOUSE_USER=admin -e CLICKHOUSE_PASSWORD=inlong -e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 clickhouse/clickhouse-server:22.8 +``` + +## Cluster Initialize +When all containers are successfully started, you can access the Inlong dashboard address http://localhost, and use the following default account to log in. +``` +User: admin +Password: inlong +``` + +### Create Cluster Tag +Click [Clusters] -> [ClusterTags] -> [Create] on the page to specify the cluster label name and person in charge. + + +:::caution +Since each component reports the ClusterTags as `default_cluster` by default, do not use other names. +::: + +### Register Pulsar Cluster +Click [Clusters] -> [ClusterTags] -> [Create] on the page to register Pulsar Cluster. + + +:::note +The ClusterTags selects the newly created `default_cluster`, the Pulsar cluster deployed by docker: + +Service URL is `pulsar://pulsar:6650`, Admin URL is `http://pulsar:8080`. +::: + +### Register Clickhouse DataNodes +Click [DataNodes] -> [Create] on the page to register ClickHouse DataNodes. + + +## Create Task +### Create Data Streams Group +Click [Ingestion] -> [Create], inpit ID and select Pulsar Type. + + +:::note +If you want to use Kafka, you need to modify the MQ_TYPE in docker-compose.yml +::: + +### Create Data Stream +Click [Next] -> [Create], config data stream. +![Create datast
[GitHub] [inlong-website] dockerzhang merged pull request #838: [INLONG-822][Doc] Add the quick start guide for collecting file and writing to ClickHouse
dockerzhang merged PR #838: URL: https://github.com/apache/inlong-website/pull/838 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] EMsnap merged pull request #8798: [INLONG-8797][Manager][Sort] Support multiple audit ids for data sync
EMsnap merged PR #8798: URL: https://github.com/apache/inlong/pull/8798 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[inlong] branch master updated (0bb98ca943 -> ebc95f4a86)
This is an automated email from the ASF dual-hosted git repository. zirui pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/inlong.git from 0bb98ca943 [INLONG-8684][Dashboard] Fixed the Vite Server Options (server.fs.deny) can be bypassed using double forward-slash (//) (#8736) add ebc95f4a86 [INLONG-8797][Manager][Sort] Support multiple audit ids for data sync (#8798) No new revisions were added by this update. Summary of changes: .../resource/sort/DefaultSortConfigOperator.java| 1 + .../inlong/sort/iceberg/IcebergTableSink.java | 7 +-- .../apache/inlong/sort/iceberg/sink/FlinkSink.java | 15 +-- .../sink/multiple/DynamicSchemaHandleOperator.java | 6 +- .../sink/multiple/IcebergMultipleStreamWriter.java | 8 ++-- .../sink/multiple/IcebergSingleStreamWriter.java| 6 +- .../jdbc/internal/JdbcBatchingOutputFormat.java | 21 ++--- .../internal/JdbcMultiBatchingOutputFormat.java | 5 - .../jdbc/internal/TableJdbcUpsertOutputFormat.java | 11 +++ .../jdbc/table/JdbcDynamicOutputFormatBuilder.java | 15 --- .../sort/jdbc/table/JdbcDynamicTableFactory.java| 4 +++- .../sort/jdbc/table/JdbcDynamicTableSink.java | 13 + .../sort/cdc/debezium/DebeziumSourceFunction.java | 6 +- .../apache/inlong/sort/cdc/mysql/MySqlSource.java | 8 +++- .../inlong/sort/cdc/mysql/source/MySqlSource.java | 1 + .../sort/cdc/mysql/source/MySqlSourceBuilder.java | 5 + .../cdc/mysql/source/config/MySqlSourceConfig.java | 9 - .../source/config/MySqlSourceConfigFactory.java | 9 - .../mysql/table/MySqlTableInlongSourceFactory.java | 4 +++- .../sort/cdc/mysql/table/MySqlTableSource.java | 15 +++ .../table/sink/StarRocksDynamicSinkFunction.java| 5 - .../table/sink/StarRocksDynamicTableSink.java | 11 --- .../sink/StarRocksDynamicTableSinkFactory.java | 6 +- 23 files changed, 149 insertions(+), 42 deletions(-)
[GitHub] [inlong] dockerzhang merged pull request #8795: [INLONG-8794][Manager] Support add streamField and sinkField
dockerzhang merged PR #8795: URL: https://github.com/apache/inlong/pull/8795 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[inlong] branch master updated: [INLONG-8794][Manager] Support add streamField and sinkField (#8795)
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong.git
The following commit(s) were added to refs/heads/master by this push:
new f4084f307a [INLONG-8794][Manager] Support add streamField and
sinkField (#8795)
f4084f307a is described below
commit f4084f307a52475a71b87160eaed6ae6e7ac8ef8
Author: fuweng11 <76141879+fuwen...@users.noreply.github.com>
AuthorDate: Thu Aug 24 17:36:28 2023 +0800
[INLONG-8794][Manager] Support add streamField and sinkField (#8795)
---
.../inlong/manager/pojo/sink/AddFieldRequest.java | 48 +++
.../manager/service/sink/StreamSinkService.java| 10 +++
.../service/sink/StreamSinkServiceImpl.java| 38
.../service/stream/InlongStreamService.java| 9 +++
.../service/stream/InlongStreamServiceImpl.java| 48 +++
.../manager/service/stream/InlongStreamTest.java | 72 ++
.../web/controller/InlongStreamController.java | 7 +++
.../openapi/OpenInLongStreamController.java| 7 +++
8 files changed, 239 insertions(+)
diff --git
a/inlong-manager/manager-pojo/src/main/java/org/apache/inlong/manager/pojo/sink/AddFieldRequest.java
b/inlong-manager/manager-pojo/src/main/java/org/apache/inlong/manager/pojo/sink/AddFieldRequest.java
new file mode 100644
index 00..1c98992e08
--- /dev/null
+++
b/inlong-manager/manager-pojo/src/main/java/org/apache/inlong/manager/pojo/sink/AddFieldRequest.java
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.manager.pojo.sink;
+
+import io.swagger.annotations.ApiModel;
+import io.swagger.annotations.ApiModelProperty;
+import lombok.AllArgsConstructor;
+import lombok.Builder;
+import lombok.Data;
+import lombok.NoArgsConstructor;
+
+import java.util.List;
+
+/**
+ * Add field request - with stream
+ */
+@Data
+@Builder
+@NoArgsConstructor
+@AllArgsConstructor
+@ApiModel("Add field request - with stream")
+public class AddFieldRequest {
+
+@ApiModelProperty("Inlong group id")
+private String inlongGroupId;
+
+@ApiModelProperty("Inlong stream id")
+private String inlongStreamId;
+
+@ApiModelProperty("Sink field list")
+private List sinkFieldList;
+
+}
diff --git
a/inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkService.java
b/inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkService.java
index 764d2b62cb..06f97bb762 100644
---
a/inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkService.java
+++
b/inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkService.java
@@ -17,6 +17,7 @@
package org.apache.inlong.manager.service.sink;
+import org.apache.inlong.manager.dao.entity.StreamSinkEntity;
import org.apache.inlong.manager.pojo.common.PageResult;
import org.apache.inlong.manager.pojo.common.UpdateResult;
import org.apache.inlong.manager.pojo.group.InlongGroupInfo;
@@ -248,6 +249,15 @@ public interface StreamSinkService {
*/
Boolean updateAfterApprove(List sinkApproveList, String
operator);
+/**
+ * Add field info
+ *
+ * @param sinkEntity stream sink information
+ * @param sinkFieldList sink field info
+ * @return whether succeed
+ */
+boolean addFields(StreamSinkEntity sinkEntity, List
sinkFieldList);
+
/**
* Converts a statement to a sinkFields
*
diff --git
a/inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkServiceImpl.java
b/inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkServiceImpl.java
index b9c0ffe40f..31092bdb0b 100644
---
a/inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkServiceImpl.java
+++
b/inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkServiceImpl.java
@@ -24,10 +24,12 @@ import org.apache.inlong.manager.common.enums.SinkStatus;
import org.apache.inlong.manager.common.enums
[GitHub] [inlong] vernedeng commented on issue #8446: [Feature][Manager] Remove the permission check logic in Services and DAOs
vernedeng commented on issue #8446: URL: https://github.com/apache/inlong/issues/8446#issuecomment-1691411537 This issue has been fixed in #8763 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] dockerzhang merged pull request #8801: [INLONG-8800][Dashboard] Owners search component optimization
dockerzhang merged PR #8801: URL: https://github.com/apache/inlong/pull/8801 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[inlong] branch master updated: [INLONG-8800][Dashboard] Owners search component optimization (#8801)
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong.git
The following commit(s) were added to refs/heads/master by this push:
new 537f1cb885 [INLONG-8800][Dashboard] Owners search component
optimization (#8801)
537f1cb885 is described below
commit 537f1cb885854d7a42c57220af66eaf8b38f1448
Author: Lizhen <88174078+bluew...@users.noreply.github.com>
AuthorDate: Thu Aug 24 18:50:08 2023 +0800
[INLONG-8800][Dashboard] Owners search component optimization (#8801)
---
inlong-dashboard/src/ui/components/UserSelect/index.tsx | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/inlong-dashboard/src/ui/components/UserSelect/index.tsx
b/inlong-dashboard/src/ui/components/UserSelect/index.tsx
index 20c753ab02..7dc34a8b82 100644
--- a/inlong-dashboard/src/ui/components/UserSelect/index.tsx
+++ b/inlong-dashboard/src/ui/components/UserSelect/index.tsx
@@ -110,12 +110,12 @@ const UserSelect: React.FC = ({
{...rest}
options={{
...rest.options,
-requestTrigger: ['onSearch'],
+requestTrigger: ['onSearch', 'onOpen'],
requestService: name => ({
url: '/user/listAll',
method: 'POST',
data: {
-name,
+keyword: name,
},
}),
requestParams: {
[GitHub] [inlong] dockerzhang merged pull request #8792: [INLONG-8788][Dashboard] Data synchronization page optimization
dockerzhang merged PR #8792: URL: https://github.com/apache/inlong/pull/8792 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[inlong] branch master updated: [INLONG-8788][Dashboard] Data synchronization page optimization (#8792)
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong.git
The following commit(s) were added to refs/heads/master by this push:
new 092920e0c8 [INLONG-8788][Dashboard] Data synchronization page
optimization (#8792)
092920e0c8 is described below
commit 092920e0c83bb78bc79fec75ed827fe9f62b7788
Author: Lizhen <88174078+bluew...@users.noreply.github.com>
AuthorDate: Thu Aug 24 18:50:23 2023 +0800
[INLONG-8788][Dashboard] Data synchronization page optimization (#8792)
---
.../src/plugins/sources/defaults/MySQLBinlog.ts| 16 ++---
.../src/ui/components/FieldList/DetailModal.tsx| 8 +++
.../src/ui/components/GroupLogs/GroupLogs.tsx | 1 +
.../src/ui/pages/SynchronizeDetail/Info/index.tsx | 15 ++--
.../SynchronizeDetail/SyncT/SourceSinkCard.tsx | 28 ++
5 files changed, 48 insertions(+), 20 deletions(-)
diff --git a/inlong-dashboard/src/plugins/sources/defaults/MySQLBinlog.ts
b/inlong-dashboard/src/plugins/sources/defaults/MySQLBinlog.ts
index c379fb74c4..7544f3da61 100644
--- a/inlong-dashboard/src/plugins/sources/defaults/MySQLBinlog.ts
+++ b/inlong-dashboard/src/plugins/sources/defaults/MySQLBinlog.ts
@@ -141,26 +141,26 @@ export default class TubeMqSource
@FieldDecorator({
type: 'input',
-tooltip: i18n.t('meta.Sources.Db.TableWhiteListHelp'),
rules: [{ required: true }],
+tooltip: i18n.t('meta.Sources.Db.DatabaseWhiteListHelp'),
props: values => ({
disabled: values?.status === 101,
}),
-visible: values => !values?.allMigration,
})
@SyncField()
- @I18n('meta.Sources.Db.TableWhiteList')
- tableWhiteList: boolean;
+ @I18n('meta.Sources.Db.DatabaseWhiteList')
+ databaseWhiteList: string;
@FieldDecorator({
type: 'input',
-rules: [{ required: false }],
-tooltip: i18n.t('meta.Sources.Db.DatabaseWhiteListHelp'),
+tooltip: i18n.t('meta.Sources.Db.TableWhiteListHelp'),
+rules: [{ required: true }],
props: values => ({
disabled: values?.status === 101,
}),
+visible: values => !values?.allMigration,
})
@SyncField()
- @I18n('meta.Sources.Db.DatabaseWhiteList')
- databaseWhiteList: string;
+ @I18n('meta.Sources.Db.TableWhiteList')
+ tableWhiteList: boolean;
}
diff --git a/inlong-dashboard/src/ui/components/FieldList/DetailModal.tsx
b/inlong-dashboard/src/ui/components/FieldList/DetailModal.tsx
index b8975eb109..fb186dd3e9 100644
--- a/inlong-dashboard/src/ui/components/FieldList/DetailModal.tsx
+++ b/inlong-dashboard/src/ui/components/FieldList/DetailModal.tsx
@@ -211,6 +211,12 @@ const Comp: React.FC = ({ inlongGroupId,
inlongStreamId, isSource, ...mod
}
}, [getData, getSinkData, inlongStreamId, modalProps.open]);
+ useEffect(() => {
+if (inlongStreamId && modalProps.open && isSource) {
+ form.setFieldValue('fieldList', data?.fieldList);
+}
+ }, [data?.fieldList, form, inlongStreamId, isSource, modalProps.open]);
+
useEffect(() => {
if (inlongStreamId && isSource === false && modalProps.open) {
if (sinkData?.list[0]?.sinkFieldList?.length === 0) {
@@ -222,6 +228,8 @@ const Comp: React.FC = ({ inlongGroupId,
inlongStreamId, isSource, ...mod
fieldType: '',
})),
});
+ } else {
+form.setFieldValue('sinkFieldList', sinkData?.list[0]?.sinkFieldList);
}
}
}, [data, form, inlongStreamId, isSource, modalProps.open, sinkData?.list]);
diff --git a/inlong-dashboard/src/ui/components/GroupLogs/GroupLogs.tsx
b/inlong-dashboard/src/ui/components/GroupLogs/GroupLogs.tsx
index ed170223ef..9cac940be8 100644
--- a/inlong-dashboard/src/ui/components/GroupLogs/GroupLogs.tsx
+++ b/inlong-dashboard/src/ui/components/GroupLogs/GroupLogs.tsx
@@ -83,6 +83,7 @@ const Comp: React.FC = ({
data: {
remark: '',
},
+timeout: 3,
});
await getData(inlongGroupId);
message.success(t('pages.GroupDashboard.ExecutionLogModal.Re-executingSuccess'));
diff --git a/inlong-dashboard/src/ui/pages/SynchronizeDetail/Info/index.tsx
b/inlong-dashboard/src/ui/pages/SynchronizeDetail/Info/index.tsx
index ee4c5b0e0d..2bceedd3cc 100644
--- a/inlong-dashboard/src/ui/pages/SynchronizeDetail/Info/index.tsx
+++ b/inlong-dashboard/src/ui/pages/SynchronizeDetail/Info/index.tsx
@@ -17,15 +17,16 @@
* under the License.
*/
-import React, { useMemo, useImperativeHandle, forwardRef, useState } from
'react';
+import React, { useMemo, useImperativeHandle, forwardRef, useState, useEffect
} from 'react';
import { Button, Space, message } from 'antd';
import FormGenerator, { useForm } from '@/ui/components/FormGenerator';
-import { useRequest, useBoolean } from '@/ui/hooks';
+import { useRequest, useBoolean, useSelector } from '@/ui/hooks';
import { useTranslation } from 'react-i18next'
