[PR] [INLONG-11539][Dashboard] Dirty data query date format modification [inlong]
wohainilaodou opened a new pull request, #11540: URL: https://github.com/apache/inlong/pull/11540 Fixes #11539 ### Motivation Dirty data query date format modification ### Modifications Dirty data query date format modification ### Verifying this change before：   after：   -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(inlong) branch master updated: [INLONG-11531][Manager] Fix bug in DolphinScheduler engine (#11532)
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong.git
The following commit(s) were added to refs/heads/master by this push:
new ace3362d6f [INLONG-11531][Manager] Fix bug in DolphinScheduler engine
(#11532)
ace3362d6f is described below
commit ace3362d6f94fcc3b913d935791b4e66f06f6d65
Author: emptyOVO <118812562+empty...@users.noreply.github.com>
AuthorDate: Mon Nov 25 18:48:32 2024 +0800
[INLONG-11531][Manager] Fix bug in DolphinScheduler engine (#11532)
---
.../dolphinscheduler/DolphinScheduleConstants.java | 3 ++
.../dolphinscheduler/DolphinScheduleEngine.java| 2 ++
.../dolphinscheduler/DolphinScheduleUtils.java | 40 +++---
.../exception/DolphinScheduleException.java| 1 +
.../DolphinScheduleEngineTest.java | 1 -
5 files changed, 42 insertions(+), 5 deletions(-)
diff --git
a/inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleConstants.java
b/inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleConstants.java
index 89dcda5b77..1488ca1fe8 100644
---
a/inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleConstants.java
+++
b/inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleConstants.java
@@ -22,6 +22,7 @@ public class DolphinScheduleConstants {
// DS public constants
public static final String DS_ID = "id";
public static final String DS_CODE = "code";
+public static final String DS_SUCCESS = "success";
public static final String DS_TOKEN = "token";
public static final String DS_PAGE_SIZE = "pageSize";
public static final String DS_PAGE_NO = "pageNo";
@@ -29,6 +30,8 @@ public class DolphinScheduleConstants {
public static final String DS_RESPONSE_DATA = "data";
public static final String DS_RESPONSE_NAME = "name";
public static final String DS_RESPONSE_TOTAL_LIST = "totalList";
+public static final int DS_DEFAULT_RETRY_TIMES = 3;
+public static final int DS_DEFAULT_WAIT_MILLS = 1000;
public static final String DS_DEFAULT_PAGE_SIZE = "10";
public static final String DS_DEFAULT_PAGE_NO = "1";
public static final String DS_DEFAULT_TIMEZONE_ID = "Asia/Shanghai";
diff --git
a/inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleEngine.java
b/inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleEngine.java
index dd0c6d0c81..5123068eab 100644
---
a/inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleEngine.java
+++
b/inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleEngine.java
@@ -132,6 +132,7 @@ public class DolphinScheduleEngine implements
ScheduleEngine {
@Override
@VisibleForTesting
public boolean handleRegister(ScheduleInfo scheduleInfo) {
+start();
String processDefUrl = projectBaseUrl + "/" + projectCode +
DS_PROCESS_URL;
String scheduleUrl = projectBaseUrl + "/" + projectCode +
DS_SCHEDULE_URL;
String processName = scheduleInfo.getInlongGroupId() +
DS_DEFAULT_PROCESS_NAME;
@@ -191,6 +192,7 @@ public class DolphinScheduleEngine implements
ScheduleEngine {
@Override
@VisibleForTesting
public boolean handleUnregister(String groupId) {
+start();
String processName = groupId + DS_DEFAULT_PROCESS_NAME;
String processDefUrl = projectBaseUrl + "/" + projectCode +
DS_PROCESS_URL;
diff --git
a/inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleUtils.java
b/inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleUtils.java
index 87cb1c5127..5fd6dd3629 100644
---
a/inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleUtils.java
+++
b/inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleUtils.java
@@ -57,11 +57,13 @@ import java.util.stream.StreamSupport;
import static
org.apache.inlong.manager.schedule.dolphinscheduler.DolphinScheduleConstants.DS_CODE;
import static
org.apache.inlong.manager.schedule.dolphinscheduler.DolphinScheduleConstants.DS_DEFAULT_PAGE_NO;
import static
org.apache.inlong.manager.schedule.dolphinscheduler.DolphinScheduleConstants.DS_DEFAULT_PAGE_SIZE;
+import static
org.apache.inlong.manager.schedule.dolphinscheduler.DolphinScheduleConstants.DS_DEFAULT_RETRY_TIMES;
import static
org.apache.inlong.manag
Re: [PR] [INLONG-11531][Manager] Fix bug in DolphinScheduler engine [inlong]
dockerzhang merged PR #11532: URL: https://github.com/apache/inlong/pull/11532 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
Zkplo commented on code in PR #11545:
URL: https://github.com/apache/inlong/pull/11545#discussion_r1856933197
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/airflow/config/AirflowConfig.java:
##
@@ -68,6 +72,15 @@ public class AirflowConfig extends ClientConfiguration {
@Value("${schedule.engine.airflow.baseUrl:http://localhost:8080/}";)
private String baseUrl;
+@PostConstruct
+public void init() {
+if (!StringUtil.isEmpty(inlongManagerUrl)) {
+String[] urlInfo = inlongManagerUrl.split(InlongConstants.COLON);
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
Zkplo commented on code in PR #11545:
URL: https://github.com/apache/inlong/pull/11545#discussion_r1856933545
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleEngine.java:
##
@@ -86,10 +84,11 @@ public void init() {
this.projectBaseUrl = dolphinUrl + DS_PROJECT_URL;
}
-public DolphinScheduleEngine(String host, int port, String username,
String password, String dolphinUrl,
+public DolphinScheduleEngine(String inlongManagerHost, int
inlongManagerPort, String username, String password,
+String dolphinUrl,
String token) {
-this.host = host;
-this.port = port;
+this.inlongManagerUrl =
+
inlongManagerHost.concat(InlongConstants.COLON).concat(String.valueOf(inlongManagerPort));
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
Zkplo commented on code in PR #11545:
URL: https://github.com/apache/inlong/pull/11545#discussion_r1856932739
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/airflow/config/AirflowConfig.java:
##
@@ -68,6 +72,15 @@ public class AirflowConfig extends ClientConfiguration {
@Value("${schedule.engine.airflow.baseUrl:http://localhost:8080/}";)
private String baseUrl;
+@PostConstruct
+public void init() {
+if (!StringUtil.isEmpty(inlongManagerUrl)) {
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] [INLONG-11541][Docker] Support Manager config volume [inlong]
emptyOVO opened a new pull request, #11542: URL: https://github.com/apache/inlong/pull/11542 Fixes #11541 ### Motivation by now, config of manager can't be modified outside, makes it not easy to change unless modified inside the container ### Modifications ### Verifying this change *(Please pick either of the following options)* - [x] This change is a trivial rework/code cleanup without any test coverage. - [ ] This change is already covered by existing tests, such as: *(please describe tests)* - [ ] This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a follow-up issue for adding the documentation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
Zkplo commented on code in PR #11545:
URL: https://github.com/apache/inlong/pull/11545#discussion_r1856855834
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleEngine.java:
##
@@ -56,10 +56,10 @@ public class DolphinScheduleEngine implements
ScheduleEngine {
private static final Logger LOGGER =
LoggerFactory.getLogger(DolphinScheduleEngine.class);
-@Value("${schedule.engine.inlong.manager.host:127.0.0.1}")
-private String host;
+@Value("${schedule.engine.inlong.manager.host:127.0.0.1:8083}")
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
fuweng11 commented on code in PR #11545:
URL: https://github.com/apache/inlong/pull/11545#discussion_r1856682992
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/airflow/config/AirflowConfig.java:
##
@@ -38,7 +38,7 @@
@EqualsAndHashCode(callSuper = true)
public class AirflowConfig extends ClientConfiguration {
-@Value("${schedule.engine.inlong.manager.host:127.0.0.1}")
+@Value("${schedule.engine.inlong.manager.url:127.0.0.1}")
Review Comment:
The URL should include port information. Does this parameter meet the
definition?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
fuweng11 commented on code in PR #11545:
URL: https://github.com/apache/inlong/pull/11545#discussion_r1856842870
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleEngine.java:
##
@@ -84,6 +84,9 @@ public class DolphinScheduleEngine implements ScheduleEngine {
@PostConstruct
public void init() {
this.projectBaseUrl = dolphinUrl + DS_PROJECT_URL;
+String[] urlInfo = managerUrl.split(":");
Review Comment:
Ditto.
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleEngine.java:
##
@@ -84,6 +84,9 @@ public class DolphinScheduleEngine implements ScheduleEngine {
@PostConstruct
public void init() {
this.projectBaseUrl = dolphinUrl + DS_PROJECT_URL;
+String[] urlInfo = managerUrl.split(":");
Review Comment:
```suggestion
String[] urlInfo = managerUrl.split(InlongConstants.COLON);
```
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/airflow/AirflowScheduleEngine.java:
##
@@ -96,9 +96,10 @@ private void initConnection() throws Exception {
AirflowResponse response = serverClient.sendRequest(
new AirflowConnectionGetter(airflowConfig.getConnectionId()));
if (!response.isSuccess()) {
+String[] urlInfo = airflowConfig.getManagerUrl().split(":");
Review Comment:
Need to check the URL, otherwise NPE will occur.
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/airflow/AirflowScheduleEngine.java:
##
@@ -96,9 +96,10 @@ private void initConnection() throws Exception {
AirflowResponse response = serverClient.sendRequest(
new AirflowConnectionGetter(airflowConfig.getConnectionId()));
if (!response.isSuccess()) {
+String[] urlInfo = airflowConfig.getManagerUrl().split(":");
Review Comment:
```suggestion
String[] urlInfo =
airflowConfig.getManagerUrl().split(InlongConstants.COLON);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
Zkplo commented on code in PR #11545:
URL: https://github.com/apache/inlong/pull/11545#discussion_r1856721594
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/airflow/config/AirflowConfig.java:
##
@@ -38,7 +38,7 @@
@EqualsAndHashCode(callSuper = true)
public class AirflowConfig extends ClientConfiguration {
-@Value("${schedule.engine.inlong.manager.host:127.0.0.1}")
+@Value("${schedule.engine.inlong.manager.url:127.0.0.1}")
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
dockerzhang commented on code in PR #11545:
URL: https://github.com/apache/inlong/pull/11545#discussion_r1856793991
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleEngine.java:
##
@@ -56,10 +56,10 @@ public class DolphinScheduleEngine implements
ScheduleEngine {
private static final Logger LOGGER =
LoggerFactory.getLogger(DolphinScheduleEngine.class);
-@Value("${schedule.engine.inlong.manager.host:127.0.0.1}")
-private String host;
+@Value("${schedule.engine.inlong.manager.host:127.0.0.1:8083}")
Review Comment:
```suggestion
@Value("${schedule.engine.inlong.manager.url:127.0.0.1:8083}")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
Zkplo opened a new pull request, #11545: URL: https://github.com/apache/inlong/pull/11545 Fixes #11544 ### Motivation ### Modifications - Simplify Annotations. - Modify the configuration key value. - Add default values. ### Verifying this change *(Please pick either of the following options)* - [x] This change is a trivial rework/code cleanup without any test coverage. - [ ] This change is already covered by existing tests, such as: *(please describe tests)* - [ ] This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
fuweng11 commented on code in PR #11545:
URL: https://github.com/apache/inlong/pull/11545#discussion_r1856913120
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleEngine.java:
##
@@ -86,10 +84,11 @@ public void init() {
this.projectBaseUrl = dolphinUrl + DS_PROJECT_URL;
}
-public DolphinScheduleEngine(String host, int port, String username,
String password, String dolphinUrl,
+public DolphinScheduleEngine(String inlongManagerHost, int
inlongManagerPort, String username, String password,
+String dolphinUrl,
String token) {
-this.host = host;
-this.port = port;
+this.inlongManagerUrl =
+
inlongManagerHost.concat(InlongConstants.COLON).concat(String.valueOf(inlongManagerPort));
Review Comment:
```suggestion
this.inlongManagerUrl = inlongManagerHost + InlongConstants.COLON +
String.valueOf(inlongManagerPort);
```
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/airflow/config/AirflowConfig.java:
##
@@ -68,6 +72,15 @@ public class AirflowConfig extends ClientConfiguration {
@Value("${schedule.engine.airflow.baseUrl:http://localhost:8080/}";)
private String baseUrl;
+@PostConstruct
+public void init() {
+if (!StringUtil.isEmpty(inlongManagerUrl)) {
Review Comment:
```suggestion
if (!StringUtil.isNotBlank(inlongManagerUrl)) {
```
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/airflow/config/AirflowConfig.java:
##
@@ -68,6 +72,15 @@ public class AirflowConfig extends ClientConfiguration {
@Value("${schedule.engine.airflow.baseUrl:http://localhost:8080/}";)
private String baseUrl;
+@PostConstruct
+public void init() {
+if (!StringUtil.isEmpty(inlongManagerUrl)) {
+String[] urlInfo = inlongManagerUrl.split(InlongConstants.COLON);
Review Comment:
Need to check the URL, otherwise NPE will occur when the `urlInfo.length` is
not equal 2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
Zkplo commented on code in PR #11545:
URL: https://github.com/apache/inlong/pull/11545#discussion_r1857848675
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleEngine.java:
##
@@ -86,10 +84,10 @@ public void init() {
this.projectBaseUrl = dolphinUrl + DS_PROJECT_URL;
}
-public DolphinScheduleEngine(String host, int port, String username,
String password, String dolphinUrl,
+public DolphinScheduleEngine(String inlongManagerHost, int
inlongManagerPort, String username, String password,
+String dolphinUrl,
String token) {
-this.host = host;
-this.port = port;
+this.inlongManagerUrl = inlongManagerHost + InlongConstants.COLON +
inlongManagerPort;
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [INLONG-11541][Docker] Support Manager config volume [inlong]
emptyOVO closed pull request #11542: [INLONG-11541][Docker] Support Manager config volume URL: https://github.com/apache/inlong/pull/11542 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-11539][Dashboard] Dirty data query date format modification [inlong]
dockerzhang merged PR #11540: URL: https://github.com/apache/inlong/pull/11540 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(inlong) branch master updated: [INLONG-11539][Dashboard] Dirty data query date format modification (#11540)
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong.git
The following commit(s) were added to refs/heads/master by this push:
new 63a1173c66 [INLONG-11539][Dashboard] Dirty data query date format
modification (#11540)
63a1173c66 is described below
commit 63a1173c66a12860e151cc4803a9cfbcb5c586da
Author: kamianlaida <165994047+wohainilao...@users.noreply.github.com>
AuthorDate: Tue Nov 26 15:22:27 2024 +0800
[INLONG-11539][Dashboard] Dirty data query date format modification (#11540)
---
.../src/ui/pages/common/DirtyModal/index.tsx | 18 ++
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/inlong-dashboard/src/ui/pages/common/DirtyModal/index.tsx
b/inlong-dashboard/src/ui/pages/common/DirtyModal/index.tsx
index d77d5829fc..402e6dbe7e 100644
--- a/inlong-dashboard/src/ui/pages/common/DirtyModal/index.tsx
+++ b/inlong-dashboard/src/ui/pages/common/DirtyModal/index.tsx
@@ -100,8 +100,8 @@ const Comp: React.FC = ({ ...modalProps }) => {
dataCount: 10,
dirtyType: '',
serverType: '',
-startTime: dayjs().format('MMDD'),
-endTime: dayjs().format('MMDD'),
+startTime: dayjs().format('MMDDHH'),
+endTime: dayjs().format('MMDDHH'),
};
const defaultTrendOptions = {
dataTimeUnit: 'D',
@@ -172,8 +172,8 @@ const Comp: React.FC = ({ ...modalProps }) => {
method: 'POST',
data: {
...options,
-startTime: options.startTime ?
dayjs(options.startTime).format('MMDD') : '',
-endTime: options.endTime ? dayjs(options.endTime).format('MMDD') :
'',
+startTime: options.startTime ?
dayjs(options.startTime).format('MMDDHH') : '',
+endTime: options.endTime ? dayjs(options.endTime).format('MMDDHH')
: '',
dataCount: form1.getFieldValue('dataCount') || 10,
keyword: form1.getFieldValue('keyword') || '',
sinkIdList: [modalProps.id],
@@ -286,7 +286,8 @@ const Comp: React.FC = ({ ...modalProps }) => {
initialValue: dayjs(options.startTime),
props: {
allowClear: true,
-format: 'MMDD',
+showTime: true,
+format: '-MM-DD HH',
},
rules: [
{ required: true },
@@ -310,7 +311,8 @@ const Comp: React.FC = ({ ...modalProps }) => {
props: values => {
return {
allowClear: true,
- format: 'MMDD',
+ showTime: true,
+ format: '-MM-DD HH',
};
},
rules: [
@@ -408,7 +410,7 @@ const Comp: React.FC = ({ ...modalProps }) => {
return {
allowClear: true,
showTime: values.dataTimeUnit === 'H',
- format: values.dataTimeUnit === 'D' ? 'MMDD' : 'MMDDHH',
+ format: values.dataTimeUnit === 'D' ? '-MM-DD' : '-MM-DD HH',
};
},
initialValue: dayjs(trendOptions.startTime),
@@ -435,7 +437,7 @@ const Comp: React.FC = ({ ...modalProps }) => {
return {
allowClear: true,
showTime: values.dataTimeUnit === 'H',
- format: values.dataTimeUnit === 'D' ? 'MMDD' : 'MMDDHH',
+ format: values.dataTimeUnit === 'D' ? '-MM-DD' : '-MM-DD HH',
};
},
rules: [
Re: [PR] [INLONG-11494][Sort] Add Kafka connector on Flink 1.18 [inlong]
aloyszhang merged PR #11501: URL: https://github.com/apache/inlong/pull/11501 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
fuweng11 commented on code in PR #11545:
URL: https://github.com/apache/inlong/pull/11545#discussion_r1857707961
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/airflow/config/AirflowConfig.java:
##
@@ -68,6 +72,17 @@ public class AirflowConfig extends ClientConfiguration {
@Value("${schedule.engine.airflow.baseUrl:http://localhost:8080/}";)
private String baseUrl;
+@PostConstruct
+public void init() {
+if (StringUtil.isNotBlank(inlongManagerUrl)) {
+String[] urlInfo = inlongManagerUrl.split(InlongConstants.COLON);
+if (urlInfo.length == 2) {
+this.inlongManagerHost = urlInfo[0];
+this.inlongManagerPort = Integer.parseInt(urlInfo[1]);
+}
Review Comment:
```suggestion
try {
if (StringUtil.isNotBlank(inlongManagerUrl)) {
String[] urlInfo =
inlongManagerUrl.split(InlongConstants.COLON);
if (urlInfo.length == 2) {
this.inlongManagerHost = urlInfo[0];
this.inlongManagerPort = Integer.parseInt(urlInfo[1]);
}
}
LOGGER.info("Init AirflowConfig success for manager url ={}",
this.inlongManagerUrl);
} catch (Exception e) {
LOGGER.error("Init AirflowConfig failed for manager url={}: ",
this.inlongManagerUrl, e);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [INLONG-7663][Manager] Support agent report new fields and automatically issue tasks [inlong]
fuweng11 closed pull request #7664: [INLONG-7663][Manager] Support agent report new fields and automatically issue tasks URL: https://github.com/apache/inlong/pull/7664 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Feature][Tool] Support parallel compilation and packaging [inlong]
aloyszhang commented on issue #7784: URL: https://github.com/apache/inlong/issues/7784#issuecomment-2499525962 #11440 support parallel building -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-7784][Tool] Support parallel compilation and packaging [inlong]
aloyszhang closed pull request #9116: [INLONG-7784][Tool] Support parallel compilation and packaging URL: https://github.com/apache/inlong/pull/9116 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-10873][SDK] Transform support factorial function [inlong]
aloyszhang commented on PR #10874: URL: https://github.com/apache/inlong/pull/10874#issuecomment-2499528697 > Please use annotation to resolve conflicting file. Please use independent UT class file to avoid UT conflicting file. @youzhi886 please fix these problems -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-7784][Tool] Support parallel compilation and packaging [inlong]
aloyszhang commented on PR #9116: URL: https://github.com/apache/inlong/pull/9116#issuecomment-2499526466 https://github.com/apache/inlong/pull/11440 support parallel building -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-11537][Sort] Optimize the session key generation of TubeMQ Source [inlong]
aloyszhang merged PR #11538: URL: https://github.com/apache/inlong/pull/11538 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(inlong) branch master updated: [INLONG-11537][Sort] Optimize the session key generation of TubeMQ Source (#11538)
This is an automated email from the ASF dual-hosted git repository.
aloyszhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong.git
The following commit(s) were added to refs/heads/master by this push:
new bcace693b1 [INLONG-11537][Sort] Optimize the session key generation of
TubeMQ Source (#11538)
bcace693b1 is described below
commit bcace693b1e53aac32b4f5d69539fa064905069e
Author: vernedeng
AuthorDate: Tue Nov 26 10:21:54 2024 +0800
[INLONG-11537][Sort] Optimize the session key generation of TubeMQ Source
(#11538)
---
.../java/org/apache/inlong/sort/tubemq/FlinkTubeMQConsumer.java| 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git
a/inlong-sort/sort-flink/sort-flink-v1.13/sort-connectors/tubemq/src/main/java/org/apache/inlong/sort/tubemq/FlinkTubeMQConsumer.java
b/inlong-sort/sort-flink/sort-flink-v1.13/sort-connectors/tubemq/src/main/java/org/apache/inlong/sort/tubemq/FlinkTubeMQConsumer.java
index abd69f8ecb..4cbd285f17 100644
---
a/inlong-sort/sort-flink/sort-flink-v1.13/sort-connectors/tubemq/src/main/java/org/apache/inlong/sort/tubemq/FlinkTubeMQConsumer.java
+++
b/inlong-sort/sort-flink/sort-flink-v1.13/sort-connectors/tubemq/src/main/java/org/apache/inlong/sort/tubemq/FlinkTubeMQConsumer.java
@@ -69,6 +69,8 @@ public class FlinkTubeMQConsumer extends
RichParallelSourceFunction
private static final Logger LOG =
LoggerFactory.getLogger(FlinkTubeMQConsumer.class);
private static final String TUBE_OFFSET_STATE = "tube-offset-state";
+private static final String UNDERSCORE = "_";
+
/**
* The address of TubeMQ master, format eg: 127.0.0.1:8715,127.0.0.2:8715.
*/
@@ -221,7 +223,10 @@ public class FlinkTubeMQConsumer extends
RichParallelSourceFunction
messagePullConsumer =
messageSessionFactory.createPullConsumer(consumerConfig);
messagePullConsumer.subscribe(topic, streamIdSet);
String jobId = getRuntimeContext().getJobId().toString();
-messagePullConsumer.completeSubscribe(sessionKey.concat(jobId),
numTasks, true, currentOffsets);
+String attemptNumber =
String.valueOf(getRuntimeContext().getAttemptNumber());
+String startSessionKey =
sessionKey.concat(UNDERSCORE).concat(jobId).concat(UNDERSCORE).concat(attemptNumber);
+LOG.info("start to init tube mq consumer, session key={}",
startSessionKey);
+messagePullConsumer.completeSubscribe(startSessionKey, numTasks, true,
currentOffsets);
running = true;
}
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
Zkplo commented on code in PR #11545:
URL: https://github.com/apache/inlong/pull/11545#discussion_r1857750082
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/airflow/config/AirflowConfig.java:
##
@@ -68,6 +72,17 @@ public class AirflowConfig extends ClientConfiguration {
@Value("${schedule.engine.airflow.baseUrl:http://localhost:8080/}";)
private String baseUrl;
+@PostConstruct
+public void init() {
+if (StringUtil.isNotBlank(inlongManagerUrl)) {
+String[] urlInfo = inlongManagerUrl.split(InlongConstants.COLON);
+if (urlInfo.length == 2) {
+this.inlongManagerHost = urlInfo[0];
+this.inlongManagerPort = Integer.parseInt(urlInfo[1]);
+}
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
emptyOVO commented on code in PR #11545:
URL: https://github.com/apache/inlong/pull/11545#discussion_r1857795317
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleEngine.java:
##
@@ -86,10 +84,10 @@ public void init() {
this.projectBaseUrl = dolphinUrl + DS_PROJECT_URL;
}
-public DolphinScheduleEngine(String host, int port, String username,
String password, String dolphinUrl,
+public DolphinScheduleEngine(String inlongManagerHost, int
inlongManagerPort, String username, String password,
+String dolphinUrl,
String token) {
-this.host = host;
-this.port = port;
+this.inlongManagerUrl = inlongManagerHost + InlongConstants.COLON +
inlongManagerPort;
Review Comment:
direct use inlongManagerUrl may be ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
emptyOVO commented on code in PR #11545: URL: https://github.com/apache/inlong/pull/11545#discussion_r1857790638 ## inlong-manager/manager-web/src/main/resources/application-test.properties: ## @@ -102,8 +102,8 @@ dirty.log.clean.interval.minutes=5 dirty.log.retention.minutes=10 dirty.log.db.table=inlong_iceberg::dirty_data_achive_iceberg -# Please confirm it is the actual address of manager -schedule.engine.inlong.manager.host= +# Inlong Manager URL accessible by the scheduler +schedule.engine.inlong.manager.url=127.0.0.1:8083 Review Comment: url should include the protocol -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
emptyOVO commented on code in PR #11545: URL: https://github.com/apache/inlong/pull/11545#discussion_r1857790638 ## inlong-manager/manager-web/src/main/resources/application-test.properties: ## @@ -102,8 +102,8 @@ dirty.log.clean.interval.minutes=5 dirty.log.retention.minutes=10 dirty.log.db.table=inlong_iceberg::dirty_data_achive_iceberg -# Please confirm it is the actual address of manager -schedule.engine.inlong.manager.host= +# Inlong Manager URL accessible by the scheduler +schedule.engine.inlong.manager.url=127.0.0.1:8083 Review Comment: url should include the protocol -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-11544][Manager] Optimize the configuration of the Manager schedule module [inlong]
fuweng11 commented on code in PR #11545:
URL: https://github.com/apache/inlong/pull/11545#discussion_r1857854891
##
inlong-manager/manager-schedule/src/main/java/org/apache/inlong/manager/schedule/dolphinscheduler/DolphinScheduleEngine.java:
##
@@ -86,10 +83,10 @@ public void init() {
this.projectBaseUrl = dolphinUrl + DS_PROJECT_URL;
}
-public DolphinScheduleEngine(String host, int port, String username,
String password, String dolphinUrl,
+public DolphinScheduleEngine(String inlongManagerUrl, String username,
String password,
+String dolphinUrl,
String token) {
Review Comment:
```suggestion
public DolphinScheduleEngine(String inlongManagerUrl, String username,
String password,
String dolphinUrl, String token) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
