Aias00 opened a new issue, #7702: URL: https://github.com/apache/shenyu/issues/7702
### Problem / current behavior [AI token limiter admission](https://github.com/apache/shenyu/blob/66d7d954ca37d651a5ad916f988a8081157a7ab4/shenyu-plugin/shenyu-plugin-ai/shenyu-plugin-ai-token-limiter/src/main/java/org/apache/shenyu/plugin/ai/token/limiter/AiTokenLimiterPlugin.java#L105) reads the accumulated counter using Redis GET before forwarding a request. `isAllowed()` at lines 133–142 does not reserve capacity. Actual tokens are added only after usage is parsed from the response, through the separately subscribed accounting chain at lines 174–186. Consequently, a burst of slow concurrent requests sharing a rule/key can all observe the same below-limit counter and proceed before any usage is recorded. Even when the configured remaining budget is small, the entire in-flight burst can consume upstream tokens. ### Scope and priority **P2 / Medium enhancement.** Exact token usage is generally unknown at admission, so this is a budgeting/design gap rather than an assertion that a trivial atomic GET/INCR can enforce exact hard limits. Existing behavior is a completed-usage threshold and should be documented as such. ### Proposed enhancement Offer an explicit policy to bound in-flight exposure, using estimated/max-token reservations with atomic Redis admission and response reconciliation, or a configurable per-key in-flight cap. Define behavior for unknown usage, errors, cancellation, streaming responses and requests spanning a window boundary. Preserve compatibility by making any stricter admission policy explicit. ### Acceptance criteria - Describe the existing completed-usage semantics and possible concurrent overshoot. - Requests sharing a budget cannot all reserve the same capacity across gateway instances when reservation mode is enabled. - Reservations are reconciled with actual reported usage and released/expired according to a defined error/cancellation policy. - In-flight state is bounded and window expiry cannot strand permanent reservations. - Report reservation, actual usage and rejected admission distinctly so operators can tune the policy. ### Evidence / limits / related work The admission/accounting flow was inspected independently by two reviewers; this proposal does not claim a measured overshoot rate, a full Redis deployment reproduction or that exact future token usage can be predicted. Historical #6514 concerns key scope, #6513 null rule fields, #6649 counter TTL and #6515 gzip usage extraction. No reservation/concurrency-budget issue was found in the complete title baseline or targeted live search. ### Reviewed revision Apache ShenYu 2.7.2-SNAPSHOT, `master@66d7d954ca37d651a5ad916f988a8081157a7ab4`. This repository review excludes unit-test and CI issue categories. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
