[I] [Improve][Doc] Add MySQL to Iceberg example document for data sync [inlong-website]
haifxu opened a new issue, #904: URL: https://github.com/apache/inlong-website/issues/904 ### Description Add `MySQL` to the `Iceberg` example document for data sync. ### Are you willing to submit PR? - [X] Yes, I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [INLONG-904][DOC] Add MySQL to Iceberg example document for data sync [inlong-website]
haifxu opened a new pull request, #905: URL: https://github.com/apache/inlong-website/pull/905 ### Prepare a Pull Request - Fixes #904 ### Motivation Add MySQL to Iceberg example document for data sync ### Modifications Add MySQL to Iceberg example document for data sync ### Documentation Chinese：  English：  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Improve][Doc] Add MySQL to StarRocks example document for data sync [inlong-website]
haifxu opened a new issue, #906: URL: https://github.com/apache/inlong-website/issues/906 ### Description Add `MySQL` to the `StarRocks` example document for data sync. ### Are you willing to submit PR? - [X] Yes, I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-9476][Sort] Add custom function for data time transform [inlong]
EMsnap commented on code in PR #9483:
URL: https://github.com/apache/inlong/pull/9483#discussion_r1429391090
##
inlong-sort/sort-core/src/main/java/org/apache/inlong/sort/function/RoundTimestampFunction.java:
##
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.sort.function;
+
+import org.apache.flink.table.functions.ScalarFunction;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.Instant;
+import java.time.LocalDateTime;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+
+/**
+ * Round timestamp and output formatted timestamp.
+ */
+public class RoundTimestampFunction extends ScalarFunction {
+
+private static final long serialVersionUID = 1L;
+
+public static final Logger LOG =
LoggerFactory.getLogger(RoundTimestampFunction.class);
+public static final ZoneId DEFAULT_ZONE = ZoneId.systemDefault();
+private transient DateTimeFormatter formatter;
+
+/**
+ * Round timestamp and output formatted timestamp.
+ * For example, if the input timestamp is 1702610371(s), the roundTime is
600(s), and the format is "MMddHHmm",
+ * the formatted timestamp is "2023121510".
+ *
+ * @param timestamp The input timestamp in seconds.
+ * @param roundTime The round time in seconds.
+ * @param format The format of the output timestamp.
+ * @return The formatted timestamp.
+ */
+public String eval(Long timestamp, Long roundTime, String format) {
+try {
+LocalDateTime dateTime = LocalDateTime.ofInstant(
+Instant.ofEpochSecond(timestamp - timestamp % roundTime),
+DEFAULT_ZONE);
+if (formatter == null) {
+formatter = DateTimeFormatter.ofPattern(format);
+}
Review Comment:
fixed and I added a test for this case, PLAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [INLONG-902][DOC] Add MySQL to ClickHouse example document for data sync [inlong-website]
EMsnap commented on code in PR #903: URL: https://github.com/apache/inlong-website/pull/903#discussion_r1429395416 ## docs/quick_start/data_sync/mysql_clickhouse_example.md: ## @@ -0,0 +1,142 @@ +--- +title: MySQL to ClickHouse Example +sidebar_position: 1 +--- + +Here we use an example to introduce how to use Apache InLong creating `MySQL -> ClickHouse` data synchronization. + +## Resource + +- Download the latest released [Apache InLong 1.9.0 installation package](https://downloads.apache.org/inlong/1.9.0/apache-inlong-1.9.0-bin.tar.gz) +- Prepare Docker, Docker Compose environment + +## Deployment +### Install Apache InLong + +Decompression apache-inlong-1.9.0-bin.tar.gz + +``` shell +# Go to installation directory +cd docker/docker-compose +# Start +docker-compose up -d +``` + +### Install ClickHouse +```shell +docker run -d --rm --net=host --name clickhouse -e CLICKHOUSE_USER=admin -e CLICKHOUSE_PASSWORD=inlong -e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 clickhouse/clickhouse-server:22.8 +``` + +## Cluster Initialize +When all containers are successfully started, you can access the InLong dashboard address http://localhost, and use the following default account to log in. +``` +User: admin +Password: inlong +``` + +### Create Cluster Tag +Click [Clusters] -> [ClusterTags] -> [Create] on the page to specify the cluster label name and person in charge. + + + +:::caution +Since each component reports the ClusterTags as `default_cluster` by default, do not use other names. +::: + +### Register Pulsar Cluster +Click [Clusters] -> [Cluster] -> [Create] on the page to register Pulsar Cluster. + + + +:::note +The ClusterTags selects the newly created `default_cluster`, the Pulsar cluster deployed by docker: + +Service URL is `pulsar://pulsar:6650`, Admin URL is `http://pulsar:8080`. +::: + +### Register ClickHouse DataNodes + +Click [DataNodes] -> [Create] on the page to register ClickHouse DataNodes. + + Review Comment: please provide a english version for this pic -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [INLONG-906][DOC] Add MySQL to StarRocks example document for data sync [inlong-website]
haifxu opened a new pull request, #907: URL: https://github.com/apache/inlong-website/pull/907 ### Prepare a Pull Request - Fixes #906 ### Motivation Add MySQL to StarRocks example document for data sync ### Modifications Add MySQL to StarRocks example document for data sync ### Documentation Chinese:  English:  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-902][DOC] Add MySQL to ClickHouse example document for data sync [inlong-website]
EMsnap commented on code in PR #903: URL: https://github.com/apache/inlong-website/pull/903#discussion_r1429398100 ## docs/quick_start/data_sync/mysql_clickhouse_example.md: ## @@ -0,0 +1,142 @@ +--- +title: MySQL to ClickHouse Example +sidebar_position: 1 +--- + +Here we use an example to introduce how to use Apache InLong creating `MySQL -> ClickHouse` data synchronization. + +## Resource + +- Download the latest released [Apache InLong 1.9.0 installation package](https://downloads.apache.org/inlong/1.9.0/apache-inlong-1.9.0-bin.tar.gz) +- Prepare Docker, Docker Compose environment + +## Deployment +### Install Apache InLong + +Decompression apache-inlong-1.9.0-bin.tar.gz + +``` shell +# Go to installation directory +cd docker/docker-compose +# Start +docker-compose up -d +``` + +### Install ClickHouse +```shell +docker run -d --rm --net=host --name clickhouse -e CLICKHOUSE_USER=admin -e CLICKHOUSE_PASSWORD=inlong -e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 clickhouse/clickhouse-server:22.8 +``` + +## Cluster Initialize +When all containers are successfully started, you can access the InLong dashboard address http://localhost, and use the following default account to log in. +``` +User: admin +Password: inlong +``` + +### Create Cluster Tag +Click [Clusters] -> [ClusterTags] -> [Create] on the page to specify the cluster label name and person in charge. + + + +:::caution +Since each component reports the ClusterTags as `default_cluster` by default, do not use other names. +::: + +### Register Pulsar Cluster +Click [Clusters] -> [Cluster] -> [Create] on the page to register Pulsar Cluster. + + + +:::note +The ClusterTags selects the newly created `default_cluster`, the Pulsar cluster deployed by docker: + +Service URL is `pulsar://pulsar:6650`, Admin URL is `http://pulsar:8080`. +::: + +### Register ClickHouse DataNodes + +Click [DataNodes] -> [Create] on the page to register ClickHouse DataNodes. + + + +## Create Task +### Create Data Streams Group + +Click [Synchronization] → [Create] on the page and input the Group ID and Steam ID: + + + +### Create Data Source +In the data source, click [New] → [MySQL] to configure the source name, address, databases and tables information. + + + +:::note +- Please create the test.source_table database table in advance, the schema is: CREATE TABLE test.source_table (id INT PRIMARY KEY, name VARCHAR(50)); +::: + +### Create Data Sink + +In the data target, click [New] → [ClickHouse] to configure the name, library table, and created ck data node. Review Comment: library table -> table name ? ## docs/quick_start/data_sync/mysql_clickhouse_example.md: ## @@ -0,0 +1,142 @@ +--- +title: MySQL to ClickHouse Example +sidebar_position: 1 +--- + +Here we use an example to introduce how to use Apache InLong creating `MySQL -> ClickHouse` data synchronization. + +## Resource + +- Download the latest released [Apache InLong 1.9.0 installation package](https://downloads.apache.org/inlong/1.9.0/apache-inlong-1.9.0-bin.tar.gz) +- Prepare Docker, Docker Compose environment + +## Deployment +### Install Apache InLong + +Decompression apache-inlong-1.9.0-bin.tar.gz + +``` shell +# Go to installation directory +cd docker/docker-compose +# Start +docker-compose up -d +``` + +### Install ClickHouse +```shell +docker run -d --rm --net=host --name clickhouse -e CLICKHOUSE_USER=admin -e CLICKHOUSE_PASSWORD=inlong -e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 clickhouse/clickhouse-server:22.8 +``` + +## Cluster Initialize +When all containers are successfully started, you can access the InLong dashboard address http://localhost, and use the following default account to log in. +``` +User: admin +Password: inlong +``` + +### Create Cluster Tag +Click [Clusters] -> [ClusterTags] -> [Create] on the page to specify the cluster label name and person in charge. + + + +:::caution +Since each component reports the ClusterTags as `default_cluster` by default, do not use other names. +::: + +### Register Pulsar Cluster +Click [Clusters] -> [Cluster] -> [Create] on the page to register Pulsar Cluster. + + + +:::note +The ClusterTags selects the newly created `default_cluster`, the Pulsar cluster deployed by docker: + +Service URL is `pulsar://pulsar:6650`, Admin URL is `http://pulsar:8080`. +::: + +### Register ClickHouse DataNodes + +Click [DataNodes] -> [Create] on the page to register ClickHouse DataNodes. + + + +## Create Task +### Create Data Streams Group + +Click [Synch
[I] [Improve] Improve how to vote committer doc [inlong-website]
EMsnap opened a new issue, #908:
URL: https://github.com/apache/inlong-website/issues/908
### Description
improved the ${Candidate_Module_Name} module, not manager module by default
### Are you willing to submit PR?
- [X] Yes, I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
(inlong-website) 01/01: [INLONG-908] Improve how to vote committer doc make module name configurable
This is an automated email from the ASF dual-hosted git repository.
zirui pushed a commit to branch 908
in repository https://gitbox.apache.org/repos/asf/inlong-website.git
commit cf2e28e936193d909f4209570be572d3b5667645
Author: pengzirui
AuthorDate: Mon Dec 18 11:04:39 2023 +0800
[INLONG-908] Improve how to vote committer doc make module name configurable
---
community/how-to-vote-a-committer-pmc.md | 4 ++--
.../current/how-to-vote-a-committer-pmc.md| 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/community/how-to-vote-a-committer-pmc.md
b/community/how-to-vote-a-committer-pmc.md
index d2a6306ed6..cab3332406 100644
--- a/community/how-to-vote-a-committer-pmc.md
+++ b/community/how-to-vote-a-committer-pmc.md
@@ -14,7 +14,7 @@ The following is a template discussion email:
I nominate ${Candidate_Full_Name} as an InLong ${Committer or PMC Member}
-Judging from the contributions in recent months, ${Candidate_Full_Name} has
submitted many implementations[1],[2] to the project and improved the
management module for the project. During the optimization and improvement
period of the project, it is hoped that more people will participate in the
actual project optimization and improvement, to let the project more perfect
and easier to use.
+Judging from the contributions in recent months, ${Candidate_Full_Name} has
submitted many implementations[1],[2] to the project and improved the
${Candidate_Module_Name} module for the project. During the optimization and
improvement period of the project, it is hoped that more people will
participate in the actual project optimization and improvement, to let the
project more perfect and easier to use.
So I nominated ${Candidate_Full_Name} as a ${Committer or PMC Member} of the
InLong project.
@@ -33,7 +33,7 @@ The following is a template poll email:
```shell
[VOTE] ${Candidate_Full_Name} as an InLong ${Committer or PMC Member}
-Judging from the contributions in recent months, ${Candidate_Full_Name} has
submitted many implementations[1],[2],[3] to the project and improved the
management module for the project. During the optimization and improvement
period of the project, it is hoped that more people will participate in the
actual project optimization and improvement, to let the project more perfect
and easier to use.
+Judging from the contributions in recent months, ${Candidate_Full_Name} has
submitted many implementations[1],[2],[3] to the project and improved the
${Candidate_Module_Name} module for the project. During the optimization and
improvement period of the project, it is hoped that more people will
participate in the actual project optimization and improvement, to let the
project more perfect and easier to use.
I think making him a ${Committer or PMC Member} will be a recognition of his
outstanding work for InLong. So, I am happy to call VOTE to accept
${Candidate_Full_Name} as an InLong ${Committer or PMC Member}.
Voting will continue for at least 72 hours or until the required number of
votes is reached.
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/how-to-vote-a-committer-pmc.md
b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/how-to-vote-a-committer-pmc.md
index eedb0610b8..23c1261ba1 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/how-to-vote-a-committer-pmc.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/how-to-vote-a-committer-pmc.md
@@ -13,7 +13,7 @@ sidebar_position: 5
I nominate ${Candidate_Full_Name} as an InLong ${Committer or PMC Member}
-Judging from the contributions in recent months, ${Candidate_Full_Name} has
submitted many implementations[1],[2] to the project and improved the
management module for the project. During the optimization and improvement
period of the project, it is hoped that more people will participate in the
actual project optimization and improvement, to let the project more perfect
and easier to use.
+Judging from the contributions in recent months, ${Candidate_Full_Name} has
submitted many implementations[1],[2] to the project and improved the
${Candidate_Module_Name} module for the project. During the optimization and
improvement period of the project, it is hoped that more people will
participate in the actual project optimization and improvement, to let the
project more perfect and easier to use.
So I nominated ${Candidate_Full_Name} as a ${Committer or PMC Member} of the
InLong project.
@@ -32,7 +32,7 @@ So I nominated ${Candidate_Full_Name} as a ${Committer or PMC
Member} of the InL
```shell
[VOTE] ${Candidate_Full_Name} as an InLong ${Committer or PMC Member}
-Judging from the contributions in recent months, ${Candidate_Full_Name} has
submitted many implementations[1],[2],[3] to the project and improved the
management module for the project. During the optimization
(inlong-website) branch 908 created (now cf2e28e936)
This is an automated email from the ASF dual-hosted git repository. zirui pushed a change to branch 908 in repository https://gitbox.apache.org/repos/asf/inlong-website.git at cf2e28e936 [INLONG-908] Improve how to vote committer doc make module name configurable This branch includes the following new commits: new cf2e28e936 [INLONG-908] Improve how to vote committer doc make module name configurable The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
Re: [PR] [INLONG-9476][Sort] Add custom function for data time transform [inlong]
gong commented on code in PR #9483:
URL: https://github.com/apache/inlong/pull/9483#discussion_r1429448367
##
inlong-sort/sort-core/src/main/java/org/apache/inlong/sort/function/RoundTimestampFunction.java:
##
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.sort.function;
+
+import org.apache.flink.table.functions.ScalarFunction;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.Instant;
+import java.time.LocalDateTime;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Round timestamp and output formatted timestamp.
+ */
+public class RoundTimestampFunction extends ScalarFunction {
+
+private static final long serialVersionUID = 1L;
+
+public static final Logger LOG =
LoggerFactory.getLogger(RoundTimestampFunction.class);
+public static final ZoneId DEFAULT_ZONE = ZoneId.systemDefault();
+private static final Map formatters = new
HashMap<>();
+
+/**
+ * Round timestamp and output formatted timestamp.
+ * For example, if the input timestamp is 1702610371(s), the roundTime is
600(s), and the format is "MMddHHmm",
+ * the formatted timestamp is "2023121510".
+ *
+ * @param timestamp The input timestamp in seconds.
+ * @param roundTime The round time in seconds.
+ * @param format The format of the output timestamp.
+ * @return The formatted timestamp.
+ */
+public String eval(Long timestamp, Long roundTime, String format) {
+try {
+LocalDateTime dateTime = LocalDateTime.ofInstant(
+Instant.ofEpochSecond(timestamp - timestamp % roundTime),
+DEFAULT_ZONE);
+DateTimeFormatter formatter = formatters.get(format);
+if (formatter == null) {
+formatter = DateTimeFormatter.ofPattern(format);
+formatters.put(format, formatter);
+}
Review Comment:
Will it have a thread safe problem? Because using shared variables
formatters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] [Bug] Logging error for the NativeFlinkSqlParser [inlong]
github-actions[bot] commented on issue #9493: URL: https://github.com/apache/inlong/issues/9493#issuecomment-1859610430 Hello @JinsYin, thank you for opening your first issue in InLong 🧡 We will respond as soon as possible ⏳ If this is a bug report, please provide screenshots or error logs for us to reproduce your issue, so we can do our best to fix it. If you have any questions in the meantime, you can also ask us on the [InLong Discussions](https://github.com/apache/inlong/discussions) 🔍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Logging error for the NativeFlinkSqlParser [inlong]
dockerzhang commented on issue #9493: URL: https://github.com/apache/inlong/issues/9493#issuecomment-1859610758 @JinsYin please add more details abort this issue, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [INLONG-9493] Fixed incorrect logging [inlong]
JinsYin opened a new pull request, #9494: URL: https://github.com/apache/inlong/pull/9494 ### Prepare a Pull Request - Fixes #9493 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-9493][Sort] Fixed incorrect logging [inlong]
gong commented on PR #9494: URL: https://github.com/apache/inlong/pull/9494#issuecomment-1859621419 @JinsYin Thanks for your contribute -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-9476][Sort] Add custom function for data time transform [inlong]
EMsnap commented on code in PR #9483:
URL: https://github.com/apache/inlong/pull/9483#discussion_r1429553688
##
inlong-sort/sort-core/src/main/java/org/apache/inlong/sort/function/RoundTimestampFunction.java:
##
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.inlong.sort.function;
+
+import org.apache.flink.table.functions.ScalarFunction;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.time.Instant;
+import java.time.LocalDateTime;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Round timestamp and output formatted timestamp.
+ */
+public class RoundTimestampFunction extends ScalarFunction {
+
+private static final long serialVersionUID = 1L;
+
+public static final Logger LOG =
LoggerFactory.getLogger(RoundTimestampFunction.class);
+public static final ZoneId DEFAULT_ZONE = ZoneId.systemDefault();
+private static final Map formatters = new
HashMap<>();
+
+/**
+ * Round timestamp and output formatted timestamp.
+ * For example, if the input timestamp is 1702610371(s), the roundTime is
600(s), and the format is "MMddHHmm",
+ * the formatted timestamp is "2023121510".
+ *
+ * @param timestamp The input timestamp in seconds.
+ * @param roundTime The round time in seconds.
+ * @param format The format of the output timestamp.
+ * @return The formatted timestamp.
+ */
+public String eval(Long timestamp, Long roundTime, String format) {
+try {
+LocalDateTime dateTime = LocalDateTime.ofInstant(
+Instant.ofEpochSecond(timestamp - timestamp % roundTime),
+DEFAULT_ZONE);
+DateTimeFormatter formatter = formatters.get(format);
+if (formatter == null) {
+formatter = DateTimeFormatter.ofPattern(format);
+formatters.put(format, formatter);
+}
Review Comment:
For every task there is a seperate function operator initialized. So after
making formatters private there won't be thread problem I believe
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [INLONG-9493][Sort] Fixed incorrect logging [inlong]
fuweng11 commented on PR #9494: URL: https://github.com/apache/inlong/pull/9494#issuecomment-1859675520 @JinsYin Please rebase from master and resubmit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
