dmsolr opened a new pull request, #7672: URL: https://github.com/apache/shenyu/pull/7672
## Which issue is this PR closes? Closes #7645. Tracker: #7650. ## What does this PR change? `CommonPluginDataSubscriber.applySelectorDataSnapshot` / `applyRuleDataSnapshot` previously routed every incoming and every stale row through the per-row subscribe path. For each row that meant: - one `ConcurrentMap.compute` on `selectorMap` / `ruleMap` that rebuilt a full `ArrayList`, ran `List.sort(...)` and produced a fresh `List.copyOf(...)` — one merge+sort per row instead of one per plugin/selector bucket; - one match-cache invalidation cycle (`removeSelectorData(plugin, id)` + `removeEmptySelectorData(plugin)` and the rule-side equivalents) per row. With a cold-init or reconnect snapshot of a few thousand rows this dominated sync-apply CPU and allocation, even though `BaseDataCache` already exposed grouped `refreshSelectorData` / `refreshRuleData` helpers used by `onSelectorRefresh` / `onRuleRefresh`. This PR reuses that machinery on the snapshot path and adds a symmetric batch removal helper. ### `BaseDataCache` Two new package-private helpers: - `removeSelectDataBatch(List<SelectorData>)` groups the batch by `pluginName` and does one `computeIfPresent` per bucket, filtering by an id `HashSet`. Dangling selectors (blank plugin name) still match every bucket, mirroring `removeSelectData(SelectorData)`. - `removeRuleDataBatch(List<RuleData>)` groups by `selectorId` with the same one-merge-per-bucket shape. ### `CommonPluginDataSubscriber` `applySelectorDataSnapshot(namespaceId, dataList)`: 1. Compute the stale rows exactly as before (namespace filter + incoming-id exclusion). 2. One `refreshSelectorData(dataList)` — grouped by plugin, merge+sort once per plugin bucket. 3. One `removeSelectDataBatch(stale)` — grouped by plugin, single merge per bucket. 4. Iterate `dataList` and `stale` to fire per-row plugin-handler callbacks (`handlerSelector` / `removeSelector`). Callbacks are wrapped in a new `runGuarded` helper that preserves the exception semantics of `subscribeDataHandler`: rethrow as `IllegalStateException` during `InitialSyncApplication`, otherwise log and continue so the rest of the snapshot still converges. 5. Coalesce match-cache invalidation per affected `pluginName` via `invalidateSelectorMatchCacheByPlugins` — uses the broader `removeSelectorData(pluginName)` / `removeRuleData(pluginName)` variants, so both positive and negative entries under the touched plugins are cleared together. Other plugins caches are untouched. `applyRuleDataSnapshot` mirrors that shape with `refreshRuleData` / `removeRuleDataBatch` / `invalidateRuleMatchCacheByPlugins`. Plugin-handler contracts, deletion semantics, ordering by `sort`, namespace isolation and empty-snapshot cleanup are unchanged. `applyPluginDataSnapshot` is untouched: `pluginMap.put` is already O(1) and there is no per-row sort to coalesce. ### Ordering vs. concurrent incremental updates The batch cache write is a single reference swap (`selectorMap = next` / `ruleMap = next`) built from the current map — the same shape as `onSelectorRefresh`. Per-row handler callbacks run after the swap so they observe the fully merged snapshot. Concurrent `onSelectorSubscribe` calls arriving mid-snapshot can still race with the swap; that race exists on `onSelectorRefresh` today and is not widened by this change. ## Tests New unit tests in `CommonPluginDataSubscriberTest`: - `selectorSnapshotBatchesMergesAndPreservesSortOrder` / `ruleSnapshotBatchesMergesAndPreservesSortOrder` — incoming rows out of order plus a replacement for an existing id must produce a single sorted list per bucket, and every row must still hit `handlerSelector` / `handlerRule`. - `selectorSnapshotInvokesRemoveSelectorForEachStaleRow` / `ruleSnapshotInvokesRemoveRuleForEachStaleRow` — empty incoming snapshot clears only the stale rows in the target namespace, keeps other namespaces, and calls `removeSelector` / `removeRule` per stale row. - `snapshotCoalescesMatchCacheInvalidationPerAffectedPlugin` — positive and negative match-cache entries under the affected plugin are cleared; entries under an unaffected plugin survive. - `selectorSnapshotHandlerFailurePropagatesDuringInitialSync` — a throwing handler during `InitialSyncApplication.run(...)` aborts the snapshot as before. - `selectorSnapshotHandlerFailureIsLoggedOutsideInitialSync` — outside initial sync the failure is logged and the remaining rows still converge. Existing `testSelectorSnapshotReplacesRowsAndRemovesOnlyStaleNamespaceRows` and `testRuleSnapshotReplacesRowsAndRemovesOnlyStaleNamespaceRows` continue to pass unchanged, so the namespace-isolation contract is preserved. Module run: ``` ./mvnw test -pl shenyu-plugin/shenyu-plugin-base -Dtest=CommonPluginDataSubscriberTest,BaseDataCacheTest Tests run: 60, Failures: 0, Errors: 0, Skipped: 0 ./mvnw checkstyle:check -pl shenyu-plugin/shenyu-plugin-base You have 0 Checkstyle violations. ``` ## Measurement status The tracker issue asks for a before/after fixed-load comparison (1k / 10k+ configs, cold init and reconnect, business traffic running). This environment does not have a load generator, controlled upstream or JFR profiling harness, so no numeric before/after is attached here. The change is source-grounded: it removes an O(n) merge+sort per row and an O(n) match-cache invalidation per row on a path that already had a batched alternative, and it does not touch request-path code. Reviewers with a benchmark rig should be able to reproduce the sync-apply delta directly. ## Compatibility No public API change. `removeSelectDataBatch` / `removeRuleDataBatch` are package-private. Behavioural differences are limited to: - Ordering of cache-write vs. plugin-handler callbacks within a single snapshot apply — callbacks now run after the full batch is merged (matches `onSelectorRefresh`), instead of interleaved per row. - Match-cache invalidation is broader per touched plugin (all entries under that plugin are cleared, not just the entries whose selector/rule id changed). Broader invalidation cannot retain stale positive or negative results; the trade-off is a slightly higher refill cost right after a snapshot, bounded by the number of affected plugins. Neither changes routing decisions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
