hanzhe-one opened a new pull request, #7969: URL: https://github.com/apache/shenyu/pull/7969
Fixes #7932 (first stage). Concurrent requests that miss the same cache key each forward to the upstream, so a burst on one cold key produces one upstream call per request. This adds a per-gateway coordinator so the first request fills the key and later requests on the same key replay the captured response. ### Behaviour | Case | Result | | --- | --- | | Concurrent misses on one eligible key | One upstream fill; followers replay its bytes | | Leader response is not reusable (cookie, `Vary`, `private`/`no-store`/`no-cache`, 206, 304, non-2xx) | Followers fill the key themselves | | Follower waits longer than `singleFlightWaitTimeoutMillis` | Follower fills the key itself | | Waiter slots for a key are exhausted | Extra requests fill the key themselves | | Response larger than `singleFlightMaxResponseBytes` | Not shared; followers fill the key themselves | | Leader errors or is cancelled | Key released, the next request leads a new fill | | Feature disabled | Every request forwards, exactly as before | A follower never subscribes to the leader's response publisher and never shares a `DataBuffer`; it decodes its own copy of the packet into its own response. Keys are the existing response cache keys, so coalescing never widens cache eligibility or merges responses the cache keeps apart. ### Configuration Added to the cache plugin config, off by default: | Field | Default | | --- | --- | | `singleFlightEnabled` | `false` | | `singleFlightMaxKeys` | `1024` | | `singleFlightMaxWaiters` | `64` | | `singleFlightWaitTimeoutMillis` | `1000` | | `singleFlightFillTimeoutMillis` | `30000` | | `singleFlightMaxResponseBytes` | `1048576` | Coordination is per gateway instance. Replicas can still fill the same key independently; no distributed lock is taken. ### Tests `CacheSingleFlightTest` covers coalescing, the disabled path, each fallback above, distinct keys, and follower cancellation. The existing cache suite is unchanged and passes. ### Not in this PR - Stale-while-revalidate and separate freshness/stale deadlines. The issue asks for these only after the miss path is correct, and they need new fields on `CacheRuleHandle` in `shenyu-common`. - Invalidation/configuration generations preventing an old fill from repopulating an invalidated entry. The cache has no generation concept yet; adding it touches `ApplicationConfigCache` invalidation. - Benchmark comparison under #7650. ### Verification `mvn test -pl shenyu-plugin/shenyu-plugin-cache/shenyu-plugin-cache-handler` — 105 tests pass. Checkstyle and RAT pass. --- Make sure that: - [x] You have read the [contribution guidelines](https://shenyu.apache.org/community/contributor-guide). - [x] You submit test cases (unit or integration tests) that back your changes. - [x] Your local test passed `./mvnw clean install -Dmaven.javadoc.skip=true`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
