[GitHub] [inlong] Yizhou-Yang opened a new pull request, #8787: [INLONG-8786][Sort] fix doris schema change in case of multiple uri
Yizhou-Yang opened a new pull request, #8787: URL: https://github.com/apache/inlong/pull/8787 ### Prepare a Pull Request - Fixes #8786 ### Motivation A user experience exception when combining doris ddl changes with a very long fenode list. ### Modifications parse the fenodes, try to execute http request on each one, return when any succeeds. ### Verifying this change Tested using a flink job. Trivial change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] Yizhou-Yang commented on issue #8651: [Feature][Sort] Add timestamp startup mode to postgres source
Yizhou-Yang commented on issue #8651: URL: https://github.com/apache/inlong/issues/8651#issuecomment-1689454650 this proves to be a huge change, implements pg split fetcher rn, should be done this week. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] dockerzhang merged pull request #8782: [INLONG-8781][Manager] Fix exception when batch task is finished
dockerzhang merged PR #8782: URL: https://github.com/apache/inlong/pull/8782 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[inlong] branch master updated: [INLONG-8781][Manager] Fix exception when batch task is finished (#8782)
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong.git
The following commit(s) were added to refs/heads/master by this push:
new 4494611b85 [INLONG-8781][Manager] Fix exception when batch task is
finished (#8782)
4494611b85 is described below
commit 4494611b851d77b9b548e151217a1d86f77c32ae
Author: kipshi <48468934+kip...@users.noreply.github.com>
AuthorDate: Wed Aug 23 16:03:57 2023 +0800
[INLONG-8781][Manager] Fix exception when batch task is finished (#8782)
Co-authored-by: kipshi
---
.../main/java/org/apache/inlong/manager/common/enums/GroupStatus.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/enums/GroupStatus.java
b/inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/enums/GroupStatus.java
index 5b65f10335..7b22b1bd23 100644
---
a/inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/enums/GroupStatus.java
+++
b/inlong-manager/manager-common/src/main/java/org/apache/inlong/manager/common/enums/GroupStatus.java
@@ -65,11 +65,11 @@ public enum GroupStatus {
GROUP_STATE_AUTOMATON.put(APPROVE_REJECTED,
Sets.newHashSet(APPROVE_REJECTED, TO_BE_APPROVAL, DELETING));
GROUP_STATE_AUTOMATON.put(APPROVE_PASSED,
Sets.newHashSet(APPROVE_PASSED, CONFIG_ING, DELETING));
-GROUP_STATE_AUTOMATON.put(CONFIG_ING, Sets.newHashSet(CONFIG_ING,
CONFIG_FAILED, CONFIG_SUCCESSFUL));
+GROUP_STATE_AUTOMATON.put(CONFIG_ING, Sets.newHashSet(CONFIG_ING,
CONFIG_FAILED, CONFIG_SUCCESSFUL, FINISH));
GROUP_STATE_AUTOMATON.put(CONFIG_FAILED,
Sets.newHashSet(CONFIG_FAILED, CONFIG_ING, CONFIG_SUCCESSFUL,
TO_BE_APPROVAL, DELETING));
GROUP_STATE_AUTOMATON.put(CONFIG_SUCCESSFUL,
-Sets.newHashSet(CONFIG_SUCCESSFUL, TO_BE_APPROVAL, CONFIG_ING,
SUSPENDING, DELETING, FINISH));
+Sets.newHashSet(CONFIG_SUCCESSFUL, TO_BE_APPROVAL, CONFIG_ING,
SUSPENDING, DELETING));
GROUP_STATE_AUTOMATON.put(SUSPENDING, Sets.newHashSet(SUSPENDING,
SUSPENDED, CONFIG_FAILED));
GROUP_STATE_AUTOMATON.put(SUSPENDED, Sets.newHashSet(SUSPENDED,
RESTARTING, DELETING));
[GitHub] [inlong] EMsnap merged pull request #8785: [INLONG-8783][Sort] Mysql connector jdbc version should be compatible with mysql-cdc version
EMsnap merged PR #8785: URL: https://github.com/apache/inlong/pull/8785 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[inlong] branch master updated: [INLONG-8783][Sort] Mysql connector jdbc version should be compatible with mysql-cdc version (#8785)
This is an automated email from the ASF dual-hosted git repository.
zirui pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong.git
The following commit(s) were added to refs/heads/master by this push:
new 883ecba33a [INLONG-8783][Sort] Mysql connector jdbc version should be
compatible with mysql-cdc version (#8785)
883ecba33a is described below
commit 883ecba33a0eda5fec90d3975035f9d462dccea5
Author: Sting
AuthorDate: Wed Aug 23 16:17:49 2023 +0800
[INLONG-8783][Sort] Mysql connector jdbc version should be compatible with
mysql-cdc version (#8785)
---
inlong-sort/sort-flink/sort-flink-v1.13/pom.xml | 8
1 file changed, 8 insertions(+)
diff --git a/inlong-sort/sort-flink/sort-flink-v1.13/pom.xml
b/inlong-sort/sort-flink/sort-flink-v1.13/pom.xml
index e637c44086..adbaa7ae0b 100644
--- a/inlong-sort/sort-flink/sort-flink-v1.13/pom.xml
+++ b/inlong-sort/sort-flink/sort-flink-v1.13/pom.xml
@@ -48,6 +48,9 @@
2.2.1
2.2.1
1.13.6.2
+
+
+8.0.21
2.7.6
2.3.0
1.0.3
@@ -101,6 +104,11 @@
flink-connector-mysql-cdc
${flink.connector.mysql.cdc.version}
+
+mysql
+mysql-connector-java
+${mysql-connector-java.version}
+
org.apache.flink
flink-connector-jdbc_${flink.scala.binary.version}
[GitHub] [inlong] healchow commented on a diff in pull request #8787: [INLONG-8786][Sort] Fix doris schema change in case of multiple uri
healchow commented on code in PR #8787:
URL: https://github.com/apache/inlong/pull/8787#discussion_r1302734749
##
inlong-sort/sort-flink/sort-flink-v1.13/sort-connectors/doris/src/main/java/org/apache/inlong/sort/doris/schema/SchemaChangeHelper.java:
##
@@ -415,12 +417,22 @@ private String authHeader() {
private boolean executeStatement(String database, String stmt) throws
IOException {
Map param = new HashMap<>();
param.put("stmt", stmt);
-String requestUrl = String.format(SCHEMA_CHANGE_API,
options.getFenodes(), database);
-HttpPost httpPost = new HttpPost(requestUrl);
-httpPost.setHeader(HttpHeaders.AUTHORIZATION, authHeader());
-httpPost.setHeader(HttpHeaders.CONTENT_TYPE, CONTENT_TYPE_JSON);
-httpPost.setEntity(new
StringEntity(dynamicSchemaFormat.objectMapper.writeValueAsString(param)));
-return sendRequest(httpPost);
+List fenodes = Arrays.asList(options.getFenodes().split(","));
Review Comment:
Will this `options.getFenodes()` be null?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [inlong] jywang616 opened a new pull request, #8790: [INLONG-8690][Security] Upgrade junit version to 4.13.1 #8690
jywang616 opened a new pull request, #8790: URL: https://github.com/apache/inlong/pull/8790 ### [INLONG-8690][Security] Upgrade junit version to 4.13.1 #8690 - Fixes https://github.com/apache/inlong/issues/8690 ### Motivation Work around TemporaryFolder on UNIX-like systems not restricting access to created files, improving the security ### Modifications Upgrade junit to version 4.13.1 in inlong-tubemq/tubemq-connectors/tubemq-connector-flume/opm.xml ### Verifying this change - [ ] This change is a trivial rework/code cleanup without any test coverage. ### Documentation - Does this pull request introduce a new feature? no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] github-actions[bot] commented on issue #8791: [Feature] 希望tubemq-client-go 支持设置日志级别
github-actions[bot] commented on issue #8791: URL: https://github.com/apache/inlong/issues/8791#issuecomment-1690879635 Hello @smarteng, thank you for opening your first issue in InLong 🧡 We will respond as soon as possible ⏳ If this is a bug report, please provide screenshots or error logs for us to reproduce your issue, so we can do our best to fix it. If you have any questions in the meantime, you can also ask us on the [InLong Discussions](https://github.com/apache/inlong/discussions) 🔍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] dockerzhang merged pull request #8736: [INLONG-8684][Dashboard] Fixed the Vite Server Options (server.fs.deny) can be bypassed using double forward-slash (//)
dockerzhang merged PR #8736: URL: https://github.com/apache/inlong/pull/8736 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[inlong] branch master updated: [INLONG-8684][Dashboard] Fixed the Vite Server Options (server.fs.deny) can be bypassed using double forward-slash (//) (#8736)
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong.git
The following commit(s) were added to refs/heads/master by this push:
new 0bb98ca943 [INLONG-8684][Dashboard] Fixed the Vite Server Options
(server.fs.deny) can be bypassed using double forward-slash (//) (#8736)
0bb98ca943 is described below
commit 0bb98ca943bb18a16cb026d3fd96ce122f0c1170
Author: cc0924 <92106612+h092...@users.noreply.github.com>
AuthorDate: Thu Aug 24 10:21:28 2023 +0800
[INLONG-8684][Dashboard] Fixed the Vite Server Options (server.fs.deny) can
be bypassed using double forward-slash (//) (#8736)
---
inlong-dashboard/package-lock.json | 329 ++---
inlong-dashboard/package.json | 2 +-
2 files changed, 164 insertions(+), 167 deletions(-)
diff --git a/inlong-dashboard/package-lock.json
b/inlong-dashboard/package-lock.json
index 5e5b426fa5..3c996716dd 100644
--- a/inlong-dashboard/package-lock.json
+++ b/inlong-dashboard/package-lock.json
@@ -1832,10 +1832,17 @@
"resolved":
"https://registry.npmjs.org/@emotion/unitless/-/unitless-0.7.5.tgz";,
"integrity":
"sha512-OWORNpfjMsSSUBVrRBVGECkhWcULOAJz9ZW8uK9qgxD+87M7jHRcvh/A96XXNhXTLmKcoYSQtBEX7lHMO7YRwg=="
},
+"@esbuild/android-arm": {
+ "version": "0.15.18",
+ "resolved":
"https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.15.18.tgz";,
+ "integrity":
"sha512-5GT+kcs2WVGjVs7+boataCkO5Fg0y4kCjzkB5bAip7H4jfnOS3dA6KPiww9W1OEKTKeAcUVhdZGvgI65OXmUnw==",
+ "dev": true,
+ "optional": true
+},
"@esbuild/linux-loong64": {
- "version": "0.14.54",
- "resolved":
"https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.14.54.tgz";,
- "integrity":
"sha512-bZBrLAIX1kpWelV0XemxBZllyRmM6vgFQQG2GdNb+r3Fkp0FOh1NJSvekXDs7jq70k4euu1cryLMfU+mTXlEpw==",
+ "version": "0.15.18",
+ "resolved":
"https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.15.18.tgz";,
+ "integrity":
"sha512-L4jVKS82XVhw2nvzLg/19ClLWg0y27ulRwuP7lcyL6AbUWB5aPglXY3M21mauDQMDfRLs8cQmeT03r/+X3cZYQ==",
"dev": true,
"optional": true
},
@@ -3233,6 +3240,47 @@
}
}
},
+"@types/babel__core": {
+ "version": "7.1.19",
+ "resolved":
"https://registry.npmjs.org/@types/babel__core/-/babel__core-7.1.19.tgz";,
+ "integrity":
"sha512-WEOTgRsbYkvA/KCsDwVEGkd7WAr1e3g31VHQ8zy5gul/V1qKullU/BU5I68X5v7V3GnB9eotmom4v5a5gjxorw==",
+ "dev": true,
+ "requires": {
+"@babel/parser": "^7.1.0",
+"@babel/types": "^7.0.0",
+"@types/babel__generator": "*",
+"@types/babel__template": "*",
+"@types/babel__traverse": "*"
+ }
+},
+"@types/babel__generator": {
+ "version": "7.6.4",
+ "resolved":
"https://registry.npmjs.org/@types/babel__generator/-/babel__generator-7.6.4.tgz";,
+ "integrity":
"sha512-tFkciB9j2K755yrTALxD44McOrk+gfpIpvC3sxHjRawj6PfnQxrse4Clq5y/Rq+G3mrBurMax/lG8Qn2t9mSsg==",
+ "dev": true,
+ "requires": {
+"@babel/types": "^7.0.0"
+ }
+},
+"@types/babel__template": {
+ "version": "7.4.1",
+ "resolved":
"https://registry.npmjs.org/@types/babel__template/-/babel__template-7.4.1.tgz";,
+ "integrity":
"sha512-azBFKemX6kMg5Io+/rdGT0dkGreboUVR0Cdm3fz9QJWpaQGJRQXl7C+6hOTCZcMll7KFyEQpgbYI2lHdsS4U7g==",
+ "dev": true,
+ "requires": {
+"@babel/parser": "^7.1.0",
+"@babel/types": "^7.0.0"
+ }
+},
+"@types/babel__traverse": {
+ "version": "7.18.0",
+ "resolved":
"https://registry.npmjs.org/@types/babel__traverse/-/babel__traverse-7.18.0.tgz";,
+ "integrity":
"sha512-v4Vwdko+pgymgS+A2UIaJru93zQd85vIGWObM5ekZNdXCKtDYqATlEYnWgfo86Q6I1Lh0oXnksDnMU1cwmlPDw==",
+ "dev": true,
+ "requires": {
+"@babel/types": "^7.3.0"
+ }
+},
"@types/babel-core": {
"version": "6.25.7",
"resolved":
"https://registry.npmjs.org/@types/babel-core/-/babel-core-6.25.7.tgz";,
@@ -3280,47 +3328,6 @@
"integrity":
"sha512-pkPtJUUY+Vwv6B1inAz55rQvivClHJxc9aVEPPmaq2cbyeMLCiDpbKpcKyX4LAwpNGi+SHBv0tHv6+0gXv0P2A==",
"dev": true
},
-"@types/babel__core": {
- "version": "7.1.19",
- "resolved":
"https://registry.npmjs.org/@types/babel__core/-/babel__core-7.1.19.tgz";,
- "integrity":
"sha512-WEOTgRsbYkvA/KCsDwVEGkd7WAr1e3g31VHQ8zy5gul/V1qKullU/BU5I68X5v7V3GnB9eotmom4v5a5gjxorw==",
- "dev": true,
- "requires": {
-"@babel/parser": "^7.1.0",
-"@babel/types": "^7.0.0",
-"@types/babel__generator": "*",
-"@types/babel__template": "*",
-"@types/babel__traverse": "*"
- }
-},
-"@types/babel__generator": {
- "version": "7.6.4",
- "resolved":
"https://registry.npmjs.org/@types/babel__generator/-/babel__generator-7.6.4.tgz";,
-
[GitHub] [inlong] Kyofin commented on a diff in pull request #8654: [INLONG-8653][Sort] Fix the query sql for postgres cdc multiple table scenerios.
Kyofin commented on code in PR #8654:
URL: https://github.com/apache/inlong/pull/8654#discussion_r1303716609
##
inlong-sort/sort-flink/sort-flink-v1.13/sort-connectors/jdbc/src/main/java/org/apache/inlong/sort/jdbc/dialect/PostgresDialect.java:
##
@@ -38,25 +38,19 @@ public class PostgresDialect extends AbstractJdbcDialect {
private static final long serialVersionUID = 1L;
-private static final String QUERY_PRIMARY_KEY_SQL = "SELECT\n" +
-"\tstring_agg (DISTINCT t3.attname, ',') AS " + PRIMARY_KEY_COLUMN
+ ",\n" +
-"\tt4.tablename AS tableName\n" +
-"FROM\n" +
-"\tpg_constraint t1\n" +
-"INNER JOIN pg_class t2 ON t1.conrelid = t2.oid\n" +
-"INNER JOIN pg_attribute t3 ON t3.attrelid = t2.oid\n" +
-"AND array_position (t1.conkey, t3.attnum) is not null\n" +
-"INNER JOIN pg_tables t4 on t4.tablename = t2.relname\n" +
-"INNER JOIN pg_index t5 ON t5.indrelid = t2.oid\n" +
-"AND t3.attnum = ANY (t5.indkey)\n" +
-"LEFT JOIN pg_description t6 on t6.objoid = t3.attrelid\n" +
-"and t6.objsubid = t3.attnum\n" +
-"WHERE\n" +
-"\tt1.contype = 'p'\n" +
-"AND length (t3.attname) > 0\n" +
-"AND t2.oid = ?::regclass\n" +
-"group by\n" +
-"\tt4.tablename";
+private static final String QUERY_PRIMARY_KEY_SQL = "SELECT\n"
++ " string_agg(pg_attribute.attname, ',') AS pkColumn,\n"
Review Comment:
good job !
When only querying the primary key, this way is indeed more concise.
https://github.com/apache/inlong/assets/18548053/07ce5efd-b796-44a3-a2a0-edf091fc07a4";>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [inlong] bluewang opened a new pull request, #8792: [INLONG-8788][Dashboard] Data synchronization page optimization
bluewang opened a new pull request, #8792: URL: https://github.com/apache/inlong/pull/8792 ### Prepare a Pull Request *(Change the title refer to the following example)* - Title Example: [INLONG-XYZ][Component] Title of the pull request *(The following *XYZ* should be replaced by the actual [GitHub Issue](https://github.com/apache/inlong/issues) number)* - Fixes https://github.com/apache/inlong/issues/8788 ### Motivation *Explain here the context, and why you're making that change. What is the problem you're trying to solve?* ### Modifications *Describe the modifications you've done.* ### Verifying this change   -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] Kyofin commented on pull request #8792: [INLONG-8788][Dashboard] Data synchronization page optimization
Kyofin commented on PR #8792: URL: https://github.com/apache/inlong/pull/8792#issuecomment-1690905248 Hi, could you provide more description? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] dockerzhang commented on pull request #8790: [INLONG-8690][Security] Upgrade junit version to 4.13.1
dockerzhang commented on PR #8790: URL: https://github.com/apache/inlong/pull/8790#issuecomment-1690910795  The newest version is `4.13.2`, and InLong has already changed to this version, so I think this PR isn't needed now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] dockerzhang closed pull request #8790: [INLONG-8690][Security] Upgrade junit version to 4.13.1
dockerzhang closed pull request #8790: [INLONG-8690][Security] Upgrade junit version to 4.13.1 URL: https://github.com/apache/inlong/pull/8790 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] dockerzhang commented on issue #8690: [Improve][Security] TemporaryFolder on unix-like systems does not limit access to created files
dockerzhang commented on issue #8690: URL: https://github.com/apache/inlong/issues/8690#issuecomment-1690910987  The newest version is `4.13.2`, and InLong has already changed to this version, so I think this PR isn't needed now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] bluewang commented on pull request #8792: [INLONG-8788][Dashboard] Data synchronization page optimization
bluewang commented on PR #8792: URL: https://github.com/apache/inlong/pull/8792#issuecomment-1690911559 > Hi, could you provide more description? ok -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [inlong] smarteng opened a new pull request, #8793: up
smarteng opened a new pull request, #8793:
URL: https://github.com/apache/inlong/pull/8793
### Prepare a Pull Request
*(Change the title refer to the following example)*
cfg := config.New(config.WithConsumerMasters("master"),
config.WithTopics([]{"topic"}),
config.WithGroup("group"),
config.WithLogLevel("info"),
)
c, err := client.NewConsumer(cfg)
- Title Example: [INLONG-XYZ][Component] Title of the pull request
*(The following *XYZ* should be replaced by the actual [GitHub
Issue](https://github.com/apache/inlong/issues) number)*
- Fixes #8791
### Motivation
*Explain here the context, and why you're making that change. What is the
problem you're trying to solve?*
### Modifications
*Describe the modifications you've done.*
### Verifying this change
*(Please pick either of the following options)*
- [ ] This change is a trivial rework/code cleanup without any test coverage.
- [ ] This change is already covered by existing tests, such as:
*(please describe tests)*
- [ ] This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end deployment with large payloads
(10MB)*
- *Extended integration test for recovery after broker failure*
### Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs
/ not documented)
- If a feature is not applicable for documentation, explain why?
- If a feature is not documented yet in this PR, please create a follow-up
issue for adding the documentation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [inlong] vernedeng commented on a diff in pull request #8795: [INLONG-8794][Manager] Support add streamField and sinkField
vernedeng commented on code in PR #8795:
URL: https://github.com/apache/inlong/pull/8795#discussion_r1303741145
##
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkServiceImpl.java:
##
@@ -700,6 +703,39 @@ public Boolean updateAfterApprove(List
approveList, String opera
return true;
}
+@Override
+public boolean addFields(StreamSinkEntity sinkEntity, List
sinkFieldList) {
+Set existFields =
sinkFieldMapper.selectBySinkId(sinkEntity.getId()).stream()
+
.map(StreamSinkFieldEntity::getFieldName).collect(Collectors.toSet());
+
+LOGGER.debug("begin to save sink fields={}", sinkFieldList);
+if (CollectionUtils.isEmpty(sinkFieldList)) {
+return true;
+}
+List needAddFieldList = new ArrayList<>();
+for (SinkField fieldInfo : sinkFieldList) {
+if (existFields.contains(fieldInfo.getFieldName())) {
+continue;
+}
+StreamSinkFieldEntity fieldEntity =
CommonBeanUtils.copyProperties(fieldInfo,
+StreamSinkFieldEntity::new);
+if (StringUtils.isEmpty(fieldEntity.getFieldComment())) {
+fieldEntity.setFieldComment(fieldEntity.getFieldName());
+}
+fieldEntity.setInlongGroupId(sinkEntity.getInlongGroupId());
+fieldEntity.setInlongStreamId(sinkEntity.getInlongStreamId());
+fieldEntity.setSinkType(sinkEntity.getSinkType());
+fieldEntity.setSinkId(sinkEntity.getId());
+fieldEntity.setIsDeleted(InlongConstants.UN_DELETED);
+needAddFieldList.add(fieldEntity);
+}
+if (CollectionUtils.isNotEmpty(needAddFieldList)) {
+sinkFieldMapper.insertAll(needAddFieldList);
Review Comment:
ditto
##
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkServiceImpl.java:
##
@@ -700,6 +703,39 @@ public Boolean updateAfterApprove(List
approveList, String opera
return true;
}
+@Override
+public boolean addFields(StreamSinkEntity sinkEntity, List
sinkFieldList) {
+Set existFields =
sinkFieldMapper.selectBySinkId(sinkEntity.getId()).stream()
+
.map(StreamSinkFieldEntity::getFieldName).collect(Collectors.toSet());
+
+LOGGER.debug("begin to save sink fields={}", sinkFieldList);
+if (CollectionUtils.isEmpty(sinkFieldList)) {
+return true;
Review Comment:
Please add more debug log
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [inlong] fuweng11 commented on a diff in pull request #8795: [INLONG-8794][Manager] Support add streamField and sinkField
fuweng11 commented on code in PR #8795:
URL: https://github.com/apache/inlong/pull/8795#discussion_r1303846830
##
inlong-manager/manager-service/src/main/java/org/apache/inlong/manager/service/sink/StreamSinkServiceImpl.java:
##
@@ -700,6 +703,39 @@ public Boolean updateAfterApprove(List
approveList, String opera
return true;
}
+@Override
+public boolean addFields(StreamSinkEntity sinkEntity, List
sinkFieldList) {
+Set existFields =
sinkFieldMapper.selectBySinkId(sinkEntity.getId()).stream()
+
.map(StreamSinkFieldEntity::getFieldName).collect(Collectors.toSet());
+
+LOGGER.debug("begin to save sink fields={}", sinkFieldList);
+if (CollectionUtils.isEmpty(sinkFieldList)) {
+return true;
Review Comment:
The log above shows whether the field is empty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [inlong] Yizhou-Yang commented on a diff in pull request #8787: [INLONG-8786][Sort] Fix doris schema change in case of multiple uri
Yizhou-Yang commented on code in PR #8787:
URL: https://github.com/apache/inlong/pull/8787#discussion_r1303864478
##
inlong-sort/sort-flink/sort-flink-v1.13/sort-connectors/doris/src/main/java/org/apache/inlong/sort/doris/schema/SchemaChangeHelper.java:
##
@@ -415,12 +417,22 @@ private String authHeader() {
private boolean executeStatement(String database, String stmt) throws
IOException {
Map param = new HashMap<>();
param.put("stmt", stmt);
-String requestUrl = String.format(SCHEMA_CHANGE_API,
options.getFenodes(), database);
-HttpPost httpPost = new HttpPost(requestUrl);
-httpPost.setHeader(HttpHeaders.AUTHORIZATION, authHeader());
-httpPost.setHeader(HttpHeaders.CONTENT_TYPE, CONTENT_TYPE_JSON);
-httpPost.setEntity(new
StringEntity(dynamicSchemaFormat.objectMapper.writeValueAsString(param)));
-return sendRequest(httpPost);
+List fenodes = Arrays.asList(options.getFenodes().split(","));
Review Comment:
it will not, there is a precondition check during initialization:
this.fenodes = Preconditions.checkNotNull(fenodes, "fenodes is empty");
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
