Aias00 opened a new issue, #7591:
URL: https://github.com/apache/shenyu/issues/7591
## Summary
`AiProxyRealKeyResolver` resolves an AI proxy selector's real API key with a
single-flight `CompletableFuture` on a **fixed 10-thread pool with an unbounded
queue**, and it performs **no negative caching and no backoff**. If resolution
keeps failing, every subsequent call re-submits a new task and the work queue
grows without bound.
## Evidence
`shenyu-admin/src/main/java/org/apache/shenyu/admin/service/support/AiProxyRealKeyResolver.java`
```java
74: this.executorService = Executors.newFixedThreadPool(10,
75: ShenyuThreadFactory.create("ai-proxy-key-resolver",
false));
```
`Executors.newFixedThreadPool` wraps tasks in a `LinkedBlockingQueue` with
**no capacity limit**.
The failure path:
```java
final CompletableFuture<String> future = inFlight.computeIfAbsent(selectorId,
id -> CompletableFuture.supplyAsync(() -> doResolve(id),
executorService)
.whenComplete((val, ex) -> {
try {
if (Objects.nonNull(ex)) {
LOG.warn("[AiProxyRealKeyResolver] resolve
failed for selectorId={}: {}", id,
ex.getMessage());
} else {
cache.put(id, new AtomicReference<>(val));
...
}
} finally {
inFlight.remove(id);
}
}));
```
- On failure the entry is removed from `inFlight` and **nothing is written
to `cache`**, so the single-flight window closes and the very next call submits
a brand-new task.
- There is no TTL, no circuit breaker, no cap on `cache` size either — the
cache grows one entry per distinct `selectorId` forever.
Callers that can drive this in a loop: `AiProxyApiKeyServiceImpl`,
`AiProxyConnectionServiceImpl`, and `AiProxySelectorResolverInvalidator` (which
invalidates entries on selector change, forcing re-resolution).
## Impact
- A persistent failure mode (DB unreachable, selector deleted, malformed
handle) turns each caller request into a queued task. With 10 worker threads
and an unbounded queue, tasks accumulate in heap until OOM.
- Selector churn grows `cache` monotonically — an unbounded in-memory map
keyed by `selectorId`.
## Suggested fix
1. Bound the queue and fail fast instead of silently buffering:
```java
int core = 10;
ThreadPoolExecutor executor = new ThreadPoolExecutor(core, core, 0L,
TimeUnit.MILLISECONDS,
new ArrayBlockingQueue<>(1024),
ShenyuThreadFactory.create("ai-proxy-key-resolver", false),
new ThreadPoolExecutor.CallerRunsPolicy());
```
2. Add a short negative cache so failures are not retried on every call,
e.g. remember the failure timestamp per `selectorId` and skip re-resolution for
a configurable cool-down (a few seconds).
3. Bound `cache` — replace `ConcurrentHashMap<String,
AtomicReference<String>>` with a size-capped cache (Caffeine or
`WindowTinyLFUMap`), or evict entries when the selector is removed/invalidated.
## Environment
- Branch: `master` @ `a7d80b7`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
