[PR] [INLONG-11603][SDK] Bump up modules in Golang SDK [inlong]
gunli opened a new pull request, #11604: URL: https://github.com/apache/inlong/pull/11604 Fixes #11603 ### Motivation Bump up modules in Golang SDK ### Modifications - go.mod - go.sum ### Verifying this change *(Please pick either of the following options)* - [x] This change is a trivial rework/code cleanup without any test coverage. - [ ] This change is already covered by existing tests, such as: *(please describe tests)* - [ ] This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a follow-up issue for adding the documentation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [INLONG-11601][SDK] Fix data race in Golang SDK [inlong]
gunli opened a new pull request, #11602: URL: https://github.com/apache/inlong/pull/11602 Fixes #11601 ### Motivation ### Modifications ### Verifying this change *(Please pick either of the following options)* - [x] This change is a trivial rework/code cleanup without any test coverage. - [ ] This change is already covered by existing tests, such as: *(please describe tests)* - [ ] This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a follow-up issue for adding the documentation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(inlong) branch master updated: [INLONG-11599][SDK] Optimize the configuration related content in the ProxyClientConfig class (#11600)
This is an automated email from the ASF dual-hosted git repository.
gosonzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong.git
The following commit(s) were added to refs/heads/master by this push:
new 1c9dd7df9d [INLONG-11599][SDK] Optimize the configuration related
content in the ProxyClientConfig class (#11600)
1c9dd7df9d is described below
commit 1c9dd7df9db2e5679457c9cee6acca7171f0bf45
Author: Goson Zhang <4675...@qq.com>
AuthorDate: Mon Dec 16 10:06:18 2024 +0800
[INLONG-11599][SDK] Optimize the configuration related content in the
ProxyClientConfig class (#11600)
Co-authored-by: gosonzhang
---
.../plugin/sinks/filecollect/SenderManager.java| 7 +-
.../sinks/filecollect/TestSenderManager.java | 1 -
.../inlong/sdk/dataproxy/ConfigConstants.java | 7 +-
.../inlong/sdk/dataproxy/DefaultMessageSender.java | 117 +++-
.../apache/inlong/sdk/dataproxy/MessageSender.java | 3 +-
.../inlong/sdk/dataproxy/ProxyClientConfig.java| 63 -
.../sdk/dataproxy/config/ProxyConfigEntry.java | 1 +
.../sdk/dataproxy/config/ProxyConfigManager.java | 301 +++--
.../inlong/sdk/dataproxy/network/ClientMgr.java| 2 +-
.../inlong/sdk/dataproxy/network/QueueObject.java | 5 +-
.../inlong/sdk/dataproxy/network/Sender.java | 39 +--
.../sdk/dataproxy/network/SyncMessageCallable.java | 11 +-
.../sdk/dataproxy/threads/MetricWorkerThread.java | 3 +-
.../sdk/dataproxy/threads/TimeoutScanThread.java | 10 +-
14 files changed, 356 insertions(+), 214 deletions(-)
diff --git
a/inlong-agent/agent-plugins/src/main/java/org/apache/inlong/agent/plugin/sinks/filecollect/SenderManager.java
b/inlong-agent/agent-plugins/src/main/java/org/apache/inlong/agent/plugin/sinks/filecollect/SenderManager.java
index ec4502a7fb..9ac9083ad8 100755
---
a/inlong-agent/agent-plugins/src/main/java/org/apache/inlong/agent/plugin/sinks/filecollect/SenderManager.java
+++
b/inlong-agent/agent-plugins/src/main/java/org/apache/inlong/agent/plugin/sinks/filecollect/SenderManager.java
@@ -203,6 +203,7 @@ public class SenderManager {
authSecretKey);
proxyClientConfig.setTotalAsyncCallbackSize(totalAsyncBufSize);
proxyClientConfig.setAliveConnections(aliveConnectionNum);
+proxyClientConfig.setRequestTimeoutMs(maxSenderTimeout * 1000L);
proxyClientConfig.setIoThreadNum(ioThreadNum);
proxyClientConfig.setEnableBusyWait(enableBusyWait);
@@ -242,7 +243,7 @@ public class SenderManager {
message.getTotalSize(), auditVersion);
asyncSendByMessageSender(cb, message.getDataList(),
message.getGroupId(),
message.getStreamId(), message.getDataTime(),
SEQUENTIAL_ID.getNextUuid(),
-maxSenderTimeout, TimeUnit.SECONDS,
message.getExtraMap(), proxySend);
+message.getExtraMap(), proxySend);
getMetricItem(message.getGroupId(),
message.getStreamId()).pluginSendCount.addAndGet(
message.getMsgCnt());
suc = true;
@@ -270,11 +271,9 @@ public class SenderManager {
private void asyncSendByMessageSender(SendMessageCallback cb,
List bodyList, String groupId, String streamId, long
dataTime, String msgUUID,
-long timeout, TimeUnit timeUnit,
Map extraAttrMap, boolean isProxySend) throws
ProxysdkException {
sender.asyncSendMessage(cb, bodyList, groupId,
-streamId, dataTime, msgUUID,
-timeout, timeUnit, extraAttrMap, isProxySend);
+streamId, dataTime, msgUUID, extraAttrMap, isProxySend);
}
/**
diff --git
a/inlong-agent/agent-plugins/src/test/java/org/apache/inlong/agent/plugin/sinks/filecollect/TestSenderManager.java
b/inlong-agent/agent-plugins/src/test/java/org/apache/inlong/agent/plugin/sinks/filecollect/TestSenderManager.java
index 9655e757ef..508e21588f 100644
---
a/inlong-agent/agent-plugins/src/test/java/org/apache/inlong/agent/plugin/sinks/filecollect/TestSenderManager.java
+++
b/inlong-agent/agent-plugins/src/test/java/org/apache/inlong/agent/plugin/sinks/filecollect/TestSenderManager.java
@@ -96,7 +96,6 @@ public class TestSenderManager {
return null;
}).when(senderManager, "asyncSendByMessageSender", Mockito.any(),
Mockito.any(), Mockito.any(), Mockito.any(),
Mockito.anyLong(), Mockito.any(),
-Mockito.anyLong(), Mockito.any(),
Mockito.any(), Mockito.anyBoolean());
senderManager.Start();
Long offset = 0L;
diff --git
a/inlong-sdk/dataproxy-sdk/src/main/java/org/apache/inlong/sdk/dataproxy/ConfigConstants.java
b/inlong-sdk/dataproxy-sdk/src/main/java/org/apache/inlong/sdk/dataproxy/ConfigConstants.java
index 7adc5087af..0216b77c2c 100644
---
a/inlong
Re: [PR] [INLONG-11597][SDK] Optimize the generation speed of UUIDv4 [inlong]
gunli commented on PR #11598: URL: https://github.com/apache/inlong/pull/11598#issuecomment-2544329558 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-11599][SDK] Optimize the configuration related content in the ProxyClientConfig class [inlong]
gosonzhang merged PR #11600: URL: https://github.com/apache/inlong/pull/11600 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(inlong) branch master updated: [INLONG-11597][SDK] Optimize the generation speed of UUIDv4 (#11598)
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong.git
The following commit(s) were added to refs/heads/master by this push:
new 64b3531a57 [INLONG-11597][SDK] Optimize the generation speed of UUIDv4
(#11598)
64b3531a57 is described below
commit 64b3531a57f22faab2156e377f696253cadeed40
Author: Daven Du
AuthorDate: Mon Dec 16 10:15:05 2024 +0800
[INLONG-11597][SDK] Optimize the generation speed of UUIDv4 (#11598)
---
.../dataproxy-sdk-twins/dataproxy-sdk-golang/dataproxy/worker.go| 4 ++--
inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/go.mod | 2 +-
inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/go.sum | 4 ++--
inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/util/id.go | 6 +++---
4 files changed, 8 insertions(+), 8 deletions(-)
diff --git
a/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/dataproxy/worker.go
b/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/dataproxy/worker.go
index 7eb076e4db..0352052fd4 100755
--- a/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/dataproxy/worker.go
+++ b/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/dataproxy/worker.go
@@ -24,7 +24,7 @@ import (
"strconv"
"time"
- "github.com/gofrs/uuid"
+ "github.com/google/uuid"
"github.com/panjf2000/gnet/v2"
"go.uber.org/atomic"
@@ -329,7 +329,7 @@ func (w *worker) sendAsync(ctx context.Context, msg
Message, callback Callback)
}
func (w *worker) buildBatchID() string {
- u, err := uuid.NewV4()
+ u, err := uuid.NewRandom()
if err != nil {
return w.indexStr + ":" +
strconv.FormatInt(time.Now().UnixNano(), 10)
}
diff --git a/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/go.mod
b/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/go.mod
index ec9d9755ac..c09534bfe1 100755
--- a/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/go.mod
+++ b/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/go.mod
@@ -23,7 +23,7 @@ toolchain go1.21.4
require (
github.com/bwmarrin/snowflake v0.3.0
github.com/go-resty/resty/v2 v2.13.1
- github.com/gofrs/uuid v4.4.0+incompatible
+ github.com/google/uuid v1.6.0
github.com/oxtoacart/bpool v0.0.0-20190530202638-03653db5a59c
github.com/panjf2000/gnet/v2 v2.5.7
github.com/prometheus/client_golang v1.19.1
diff --git a/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/go.sum
b/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/go.sum
index cb9f6e32b2..69702aef4f 100755
--- a/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/go.sum
+++ b/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/go.sum
@@ -9,10 +9,10 @@ github.com/davecgh/go-spew v1.1.1
h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c
github.com/davecgh/go-spew v1.1.1/go.mod
h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/go-resty/resty/v2 v2.13.1
h1:x+LHXBI2nMB1vqndymf26quycC4aggYJ7DECYbiz03g=
github.com/go-resty/resty/v2 v2.13.1/go.mod
h1:GznXlLxkq6Nh4sU59rPmUw3VtgpO3aS96ORAI6Q7d+0=
-github.com/gofrs/uuid v4.4.0+incompatible
h1:3qXRTX8/NbyulANqlc0lchS1gqAVxRgsuW1YrTJupqA=
-github.com/gofrs/uuid v4.4.0+incompatible/go.mod
h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod
h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
+github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
+github.com/google/uuid v1.6.0/go.mod
h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/klauspost/compress v1.17.9
h1:6KIumPrER1LHsvBVuDa0r5xaG0Es51mhhB9BQB2qeMA=
github.com/klauspost/compress v1.17.9/go.mod
h1:Di0epgTjJY877eYKx5yC51cX2A2Vl2ibi7bDH9ttBbw=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822
h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
diff --git a/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/util/id.go
b/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/util/id.go
index e4afc5834f..268a907da0 100755
--- a/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/util/id.go
+++ b/inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-golang/util/id.go
@@ -20,7 +20,7 @@ import (
"log"
"github.com/bwmarrin/snowflake"
- "github.com/gofrs/uuid"
+ "github.com/google/uuid"
"github.com/zentures/cityhash"
)
@@ -43,12 +43,12 @@ func init() {
// UInt64UUID generates an uint64 UUID
func UInt64UUID() (uint64, error) {
- guid, err := uuid.NewV4()
+ guid, err := uuid.NewRandom()
if err != nil {
return 0, err
}
- bytes := guid.Bytes()
+ bytes := guid[:]
length := len(bytes)
return cityhash.CityHash64WithSeeds(bytes, uint32(length),
13329145742295551469, 7926974186468552394), nil
}
Re: [PR] [INLONG-11597][SDK] Optimize the generation speed of UUIDv4 [inlong]
dockerzhang merged PR #11598: URL: https://github.com/apache/inlong/pull/11598 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
