Aias00 opened a new issue, #7930: URL: https://github.com/apache/shenyu/issues/7930
## Goal Protect an upstream from rising concurrency when its latency degrades, and recover capacity gradually when it becomes healthy. Connection-pool sizing alone is not a feedback-based admission policy. ## Existing capabilities and boundary #7644 and PR #7684 concern connection-pool/pending-acquisition capacity. Reuse those limits and the existing fault-tolerance/metrics extension points; this feature should not introduce a second connection pool or replace static rate limiting. A latency controller is a proposal to evaluate, not an assertion of measured throughput gains. ## MVP - Offer an opt-in per-upstream-group controller with configured minimum/maximum concurrency, initial limit, sampling interval, minimum sample count and conservative recovery behavior. - Begin with observation mode: compute a recommended limit and expose decisions without rejecting traffic. Document the chosen feedback algorithm before enabling enforcement. - In enforcement mode, admit only within the current limit with a documented overload response. Prefer immediate rejection initially; any queue must have a finite size and deadline. - Define the measured interval, treatment of retries, timeouts, errors and cancellations, and exclusion of local admission/queue delay from upstream latency samples. - Start with non-streaming HTTP and a per-gateway limit. Explicitly document aggregate exposure across replicas; SSE/WebSocket/AI stream occupancy needs a separate policy before support. ## Acceptance criteria - [ ] Controlled healthy, slow, failing and recovering upstream scenarios demonstrate bounded in-flight work and stable convergence without sustained oscillation. - [ ] Sparse traffic, insufficient samples and controller cold starts retain a documented safe limit; changes stay within configured bounds. - [ ] Completion, error and cancellation release permits exactly once. Rule removal/configuration changes do not strand permits or grow controller state indefinitely. - [ ] Observe-only and disabled modes preserve routing behavior; enforcement reports bounded decision/rejection metrics and does not block event-loop threads. - [ ] Compare against static concurrency and existing bounded pools using #7650: successful throughput, upstream latency, gateway p99, rejection rate, CPU and memory. Report regressions as well as gains. - [ ] Document interactions with retries, circuit breakers and horizontally scaled gateways; do not claim a cluster-wide hard limit. Related: #7644, PR #7684 and #7650. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
