GitHub user AlexStocks created a discussion: [Proposal] AI era evolution directions for apache/dubbo-go
# [Proposal] AI era evolution directions for apache/dubbo-go: focus on AI-ready infrastructure, not another AI framework After a multi-round internal battle analysis against the current `develop` branch, I think the most pragmatic AI-era direction for `apache/dubbo-go` is **not** to become another AI application framework. Instead, `dubbo-go` is much better positioned to evolve into a **high-performance Go RPC + governance substrate for AI services**. This proposal tries to answer one question: > In the AI era, where should `dubbo-go` evolve so that the direction is both > meaningful and consistent with its existing strengths? --- ## 1. A boundary that matters: what `dubbo-go` should and should not become ### What `dubbo-go` should become A solid foundation for AI services in Go, including: - streaming-friendly RPC - model/service capability discovery - governance-aware routing - AI-specific observability - safe service exposure to external tools / agents - policy-driven cost / latency / tenant control ### What `dubbo-go` should **not** become It should not try to absorb into core: - prompt engineering frameworks - memory / chat history management - RAG orchestration - agent workflow runtime - vector DB integration as first-class core concern - model-provider-specific SDK logic The reason is simple: these are product-layer or ecosystem-layer concerns, while `dubbo-go` is strongest as a communication / governance / lifecycle / interoperability substrate. --- ## 2. Why this direction matches the current repo better than generic “AI integration” >From the current repository signals, `dubbo-go` already has strong foundations >around: - Triple / gRPC-compatible communication - generic invocation - service discovery / metadata / registry integration - dynamic config / route update - traffic governance - OTel / metrics / tracing - OpenAPI / codegen tooling So the best AI-related evolution is not to start from zero, but to **reinterpret these existing strengths for AI workloads**. In other words: - don’t build “AI app runtime” in core - build “AI service runtime governance” on top of existing Dubbo-go primitives --- ## 3. Suggested priority directions ## P0: AI-ready service governance ### 3.1 Model/service capability discovery Today, service discovery mostly answers: *where is the instance?* For AI workloads, consumers also need to know: - model family / model name - context window - modality (text / image / embedding / audio / tool) - region / compliance tag - latency SLA - cost tier - tenant visibility - version / rollout stage This suggests a natural evolution: - extend metadata / registry semantics for AI-capability-aware discovery - let consumers subscribe by **capability** instead of only endpoint identity This is a very natural fit for `dubbo-go`. --- ### 3.2 Governance-aware model routing Once capability metadata exists, the next step is routing. `dubbo-go` should be able to support policy-driven routing for AI workloads such as: - route by cost budget - route by latency target - route by tenant / compliance level - route by model quality tier - route by fallback policy - route by canary / experiment stage The key point here is: this should be **rule-driven and auditable**, not “let an LLM decide routing online”. That keeps the system deterministic and reviewable while still AI-aware. --- ### 3.3 AI-specific observability Traditional RPC observability is not enough for AI services. For AI traffic, we also care about: - TTFT (time to first token) - stream chunk interval - tokens in / tokens out - session / prompt / model identifiers - fallback frequency - retries caused by model/service degradation - cost attribution A strong direction for `dubbo-go` is to extend its existing observability stack so AI workloads become first-class citizens in tracing and metrics. This has immediate operational value and does not require turning the project into an AI framework. --- ## P1: Tool / Agent ecosystem enablement, but outside of agent runtime core ### 3.4 Generic invocation + schema exposure for tools One strong opportunity is to make existing Dubbo services easier to expose as machine-consumable tools. Because `dubbo-go` already has generic invocation and codegen/tooling foundations, it is well-positioned to support: - service → tool descriptor generation - schema-first service exposure - structured input/output constraints - safe metadata for external tool consumers This could make `dubbo-go` a strong backend substrate for MCP / tool-calling ecosystems **without** forcing agent orchestration into the core. --- ### 3.5 One contract, multiple consumers A practical direction would be to evolve IDL / OpenAPI / Triple tooling so one contract can serve: - RPC clients - OpenAPI consumers - AI tool consumers - structured output validators This kind of schema-first investment is lower-risk and more durable than embedding fast-moving AI product abstractions into the framework core. --- ## P2: Streaming and long-session semantics for AI traffic AI workloads stress communication differently from classic RPC: - longer-lived sessions - streaming output - cancellation sensitivity - backpressure handling - large payloads / context windows A meaningful protocol-layer evolution for `dubbo-go` would be to harden and optimize: - streaming semantics - cancellation propagation - session metadata propagation - flow control / backpressure behavior - observability around stream lifecycle This benefits AI services directly while still remaining squarely inside the RPC / protocol domain. --- ## 4. A simple principle for deciding whether an AI direction belongs in core I suggest using this filter: A direction is a good `dubbo-go` core direction if it: 1. builds on existing strengths like protocol / registry / routing / config / observability 2. helps both traditional microservices and AI services, or at least uses the same substrate cleanly 3. remains provider-neutral 4. is auditable, deterministic, and production-governance-friendly 5. does not force product-layer agent logic into framework core If a proposal depends on: - prompt templates - vector storage choices - model-vendor SDK churn - chat memory policies - agent state machines then it is probably better placed in **ecosystem projects / adapters / plugins**, not in `dubbo-go` core. --- ## 5. Recommended roadmap shape ### Phase 1: AI-ready infra Focus on: - capability metadata model - AI-aware service discovery - routing policy extensions - AI observability metrics/tags - streaming/cancellation hardening ### Phase 2: AI-integrated tooling Focus on: - tool schema generation - generic call → tool exposure - better contract/toolchain alignment for OpenAPI / Triple / AI consumers ### Phase 3: Ecosystem incubation Outside core, incubate: - MCP adapters - service-to-tool bridges - AI gateway integration patterns - operational assistants / diagnosis tooling This keeps core clean while still enabling an AI ecosystem around `dubbo-go`. --- ## 6. My concrete recommendation If the community wants one sentence to describe the direction, I would suggest: > `apache/dubbo-go` should evolve into a high-performance Go RPC and governance > substrate for AI services, rather than another AI application framework. And if we want a short prioritized list, I would rank it like this: 1. **AI capability-aware metadata and discovery** 2. **Governance-aware model/service routing** 3. **AI-specific observability** 4. **Streaming/cancellation/session hardening for AI traffic** 5. **Tool/schema exposure built on generic invocation and codegen** 6. **Keep agent/prompt/RAG orchestration in ecosystem, not core** --- ## 7. Questions for the community I think the next useful step is not immediately writing code, but aligning on boundary and priority. Questions: 1. Do we agree that the primary AI-era role of `dubbo-go` should be **AI service infrastructure**, not **AI application framework**? 2. Among metadata / routing / observability / tool exposure, which one should be the first community-level direction? 3. Should AI-related functionality land in core, or should some parts be incubated first as plugin / adapter / side project? 4. Is there already a concrete production use case from the community that can serve as the anchor scenario? If there is interest, this discussion can later be split into smaller RFCs / issues by topic. GitHub link: https://github.com/apache/dubbo-go/discussions/3454 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
