Aias00 opened a new issue, #7808:
URL: https://github.com/apache/shenyu/issues/7808
### Is there an existing issue for this?
- [x] I searched existing issues for masterCheck, WebSocket failover and
Admin restart.
- Related: #6461 reports repeated reconnection/Broken pipe after restart;
#6784 fixed orphaned clients during removal. This report isolates delayed
configuration convergence after an otherwise successful master election on
current master.
### Current Behavior
After stopping the elected Admin master in a two-Admin JDBC cluster, the
remaining Admin became master and accepted a rule update, but both gateways
continued applying the old rule until approximately the next 60-second
service-level WebSocket master check.
Measured in a real local cluster:
```text
admin-master-failover:
election_seconds = 0.156
new-admin-master-publishes-config:
convergence_seconds = 60.772
```
The election measurement starts after `docker compose stop` returns, so
0.156 s is the observed polling time to see the changed leader, not an
end-to-end crash-detection SLA. The 60.772 s measurement starts after the rule
update was accepted and ends when the changed behavior is observed on both
gateways.
Existing proxy requests remained available from cached configuration. The
problem is stale configuration after control-plane failover, not a total
data-plane outage.
### Expected Behavior
Once an Admin connection closes or is known not to be master, the gateway
should promptly re-evaluate the configured Admin endpoints and resynchronize
with the new master. The service-level master-check/failover delay should be
configurable and covered by integration tests, rather than imposing an
unconfigurable 60-second cadence.
### Steps To Reproduce
1. Start two Admin nodes sharing MySQL with JDBC cluster mode, matching
JWT/event/WebSocket secrets, and two gateways.
2. Configure both Admin WebSocket URLs on each gateway and verify initial
configuration synchronization.
3. Create a divide route backed by any reachable HTTP service. Confirm both
gateways can call it.
4. Stop the elected Admin master and observe the shared cluster_master
record change to the survivor.
5. On the surviving Admin, disable a rule using authenticated `POST
/rule/batchEnabled` and confirm the database update.
6. Poll actual route behavior on each gateway every second. In this run both
gateways continued using the enabled rule and converged after 60.772 seconds.
7. Re-enable the rule and restore the stopped Admin after the test.
### Environment
- ShenYu `2.7.2-SNAPSHOT`, master `04f7c2e8b7d282f14510157a9ab0bfaab538ecac`
- Java 17, ARM64 Linux Docker containers
- Two Admin nodes in JDBC cluster mode, one shared MySQL 8.0 database
- Two bootstrap nodes using WebSocket sync and both Admin URLs
### Code Evidence
- [WebsocketSyncDataService.java, lines
127–174](https://github.com/apache/shenyu/blob/04f7c2e8b7d282f14510157a9ab0bfaab538ecac/shenyu-sync-data-center/shenyu-sync-data-websocket/src/main/java/org/apache/shenyu/plugin/sync/data/websocket/WebsocketSyncDataService.java#L127-L174)
schedules masterCheck with a hard-coded `TimeUnit.SECONDS.toMillis(60)`.
Cleanup/removal of disconnected or non-master clients occurs in this
service-level check.
- [ShenyuWebsocketClient.java, lines
316–334](https://github.com/apache/shenyu/blob/04f7c2e8b7d282f14510157a9ab0bfaab538ecac/shenyu-sync-data-center/shenyu-sync-data-websocket/src/main/java/org/apache/shenyu/plugin/sync/data/websocket/client/ShenyuWebsocketClient.java#L316-L334):
onClose calls close and invalidates initial-sync state; it does not notify the
owning service to rediscover/reselect the master immediately.
- [healthCheck(), lines
352–377](https://github.com/apache/shenyu/blob/04f7c2e8b7d282f14510157a9ab0bfaab538ecac/shenyu-sync-data-center/shenyu-sync-data-websocket/src/main/java/org/apache/shenyu/plugin/sync/data/websocket/client/ShenyuWebsocketClient.java#L352-L377)
reconnects the same client/URL, or sends ping/instance information. Its
10-second timer is not an immediate service-level master-discovery trigger.
- masterCheck only rebuilds clients when its list is empty at method entry.
If a pass removes the last client, rebuilding may be deferred to a later pass;
the measured ~61 seconds is an observed case, not a guaranteed worst-case bound.
Admin reconciliation.interval is a separate compensation mechanism and does
not make this gateway-side master-check interval configurable.
### Suggested Scope / Validation
Evaluate a configurable service-level check interval plus a bounded,
serialized rediscovery trigger for close/non-master events. Preserve reconnect
backoff and prevent concurrent callbacks from leaking or recreating obsolete
clients.
Add an integration test that stops the elected Admin, changes configuration
through the new master, and checks application on multiple gateways within the
configured failover budget. Cover both a closed connection and a still-open
connection whose Admin has become a follower.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
