wenjin272 commented on issue #912: URL: https://github.com/apache/flink-agents/issues/912#issuecomment-5707638295
For the first question, I think the extra round trip should happen only when the effective strategy resolves to `NATIVE`. The `PROMPT` path can keep the current prompt-constrained final turn and parsing behavior. One nuance I found from LangChain is that its `ProviderStrategy` does not necessarily require an extra finalization call. When the model supports using tools and native structured output together, LangChain sends both the business tools and `response_format` on every model call, then immediately parses the response when no tool calls are returned. Its `AutoStrategy` checks this combined capability before selecting `ProviderStrategy`; otherwise, it falls back to `ToolStrategy` ([selection](https://github.com/langchain-ai/langchain/blob/master/libs/langchain_v1/langchain/agents/factory.py#L512-L555), [binding](https://github.com/langchain-ai/langchain/blob/master/libs/langchain_v1/langchain/agents/factory.py#L1231-L1285)). Ideally, our behavior could therefore be: - Native structured output can coexist with tools: pass the schema during the loop and parse the first no-tool response directly. - Native structured output cannot coexist with tools: issue the additional tool-free finalization call. - Prompt strategy: keep the current behavior without an additional call. We can first evaluate whether distinguishing the first two cases is easy to express and implement. If it adds too much complexity, I am also fine with the simpler initial implementation where every `NATIVE` path consistently uses an additional tool-free finalization call, and we optimize compatible models later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
