GitHub user weiqingy added a comment to the discussion: [Discussion] Pluggable model routing — API & design (PR #852 follow-up)
Thanks for the detailed reply. The updated direction works for me at the design-discussion level. I'm aligned with the main v1 shape: framework-owned routing in `ChatModelAction`, structured `RoutingDecision`, observability-only `ModelRoutingEvent`, route once per `initialRequestId` / ReAct loop, reuse the selected concrete model across tool rounds, and remove public `cache` / `cacheSize` from v1. I also agree that the remaining questions are mostly implementation-contract details rather than blockers for closing this discussion. A few that seem worth attention during implementation: 1. how the selected model / routing decision is persisted and replayed, especially for mid-action recovery and when no action state store is configured; 2. whether the deployable custom-strategy shape is descriptor/class-based rather than relying on live lambda/object instances that cannot cross the plan boundary; 3. whether `MODEL_ROUTER` resource registration has Java/Python/YAML parity and clear duplicate-name semantics; 4. how fallback composes with the existing fail/retry/ignore behavior in `ChatModelAction`; 5. how routing lines up with the in-flight observability work, especially model-resource health/retry metrics and trace metadata for the selected backend. One related point seems worth settling before implementation, since it shapes the SPI rather than only the implementation. If `Strategies.llm(...)` ships in v1 while its observability is deferred, the judge call likely has to sit inside a synchronous `route()`. Moving that judge call onto a framework-managed observable path later may require changing the extension point rather than just extending the implementation. So I think we should decide whether `Strategies.llm(...)` is part of v1 or explicitly a follow-up before the interface is finalized. Given the direction is now clear and the remaining items are implementation-specific, no further comments from my side on this discussion. I'm fine with closing it and continuing the detailed review on the PR. GitHub link: https://github.com/apache/flink-agents/discussions/897#discussioncomment-17800930 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
