imbajin commented on code in PR #3157:
URL: https://github.com/apache/hugegraph/pull/3157#discussion_r3895771399
##########
hugegraph-pd/hg-pd-client/src/main/java/org/apache/hugegraph/pd/client/KvClient.java:
##########
@@ -56,13 +64,37 @@
@Slf4j
public class KvClient<T extends WatchResponse> extends AbstractClient
implements Closeable {
- private AtomicLong clientId = new AtomicLong(0);
- private Semaphore semaphore = new Semaphore(1);
- private AtomicBoolean closed = new AtomicBoolean(false);
- private Set<StreamObserver> observers = ConcurrentHashMap.newKeySet();
+ private static final long RECONNECT_DELAY_MS = 1000L;
+ private static final long WATCH_START_TIMEOUT_MS = 5000L;
+ private static final Set<Status.Code> NON_RETRYABLE_WATCH_ERRORS =
+ Set.of(Status.Code.INVALID_ARGUMENT,
+ Status.Code.NOT_FOUND,
+ Status.Code.ALREADY_EXISTS,
+ Status.Code.PERMISSION_DENIED,
+ Status.Code.FAILED_PRECONDITION,
+ Status.Code.OUT_OF_RANGE,
+ Status.Code.UNIMPLEMENTED,
+ Status.Code.DATA_LOSS,
+ Status.Code.UNAUTHENTICATED);
+
+ private final AtomicLong lockClientId = new AtomicLong(0);
+ private final Semaphore lockSemaphore = new Semaphore(1);
+ private final AtomicBoolean closed = new AtomicBoolean(false);
+ private final Set<WatchSubscription> subscriptions =
ConcurrentHashMap.newKeySet();
+ private final ScheduledExecutorService reconnectExecutor;
+ private long transportGeneration;
public KvClient(PDConfig pdConfig) {
+ this(pdConfig, Executors.newSingleThreadScheduledExecutor(runnable -> {
Review Comment:
⚠️ Important. This single-thread scheduler runs `reconnect()` synchronously,
and `startWatch()` calls `streamingCall()` while holding the client monitor.
After a watch invalidates the async stub, `getStub()` enters
`AbstractClient.resetStub()`, where the blocking `getMembers()` call has the
default 60-second deadline and is repeated for every configured host. One
blackholed peer can therefore occupy the only scheduler thread for up to
`hostCount * grpcTimeOut`; other subscriptions cannot run their reconnect or
first-frame watchdog, and `KvClient.close()` waits for the same monitor. Move
transport discovery off the shared scheduler/monitor or otherwise isolate and
bound reconnect work, and add a multi-watch test with one unreachable peer and
another watch recovering.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
