[
https://issues.apache.org/jira/browse/CAMEL-23963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18099988#comment-18099988
]
Guillaume Nodet commented on CAMEL-23963:
-----------------------------------------
This issue is being investigated by a coding agent (Claude Code on behalf of
gnodet).
Analysis: Two hardcoded error-handling policies in
OpenAIProducer.processNonStreamingAgentic and OpenAIToolExecutionProducer: (1)
unknown tool names throw IllegalStateException, (2) tool execution errors are
stringified back to the model. The fix adds two configurable enum options
(toolExecutionErrorStrategy, hallucinatedToolNameStrategy) to
OpenAIConfiguration, touching the two producer classes, docs, and tests.
_Note: This comment was generated by a coding agent and requires manual
verification._
> camel-openai: configurable strategies for tool execution errors and
> hallucinated tool names in the agentic loop
> ---------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-23963
> URL: https://issues.apache.org/jira/browse/CAMEL-23963
> Project: Camel
> Issue Type: Improvement
> Components: camel-openai
> Affects Versions: 4.21.0
> Reporter: Federico Mariani
> Assignee: Guillaume Nodet
> Priority: Major
> Labels: ai
>
> The MCP agentic loop currently hard-codes two policies
> ({{OpenAIProducer.processNonStreamingAgentic}}):
> * *Tool execution errors* are always stringified ({{"Error: Tool execution
> failed: " + e.getMessage()}}) and fed back to the model. This is a sane
> default, but (a) the Camel error handler ({{onException}}, dead-letter) can
> never observe tool failures, and (b) raw exception messages may leak internal
> details (connection strings, hostnames) to a third-party LLM provider — a
> concern the CAMEL-23382 design doc calls out explicitly.
> * *Unknown/hallucinated tool names* always throw {{IllegalStateException}},
> failing the exchange. Models do hallucinate tool names; the alternative
> strategy — returning a corrective tool result listing the available tools so
> the model retries — is what LangChain4j implements
> ({{HallucinatedToolNameStrategy}}, cf. CAMEL-23928) and usually converges in
> one extra round trip.
> Proposal: two endpoint options, names kept vendor-neutral so they can survive
> the migration to the unified tool abstraction (CAMEL-23382 step 6 will
> re-route dispatch through {{AiToolExecutor}}/{{AiToolResult}}, whose sealed
> result type distinguishes {{ArgumentError}} from {{ExecutionError}} and
> explicitly leaves the policy to each component):
> * {{toolExecutionErrorStrategy=repromptModel|failExchange}} (default
> {{repromptModel}}, current behavior). {{failExchange}} sets the tool
> exception on the exchange so standard Camel error handling fires — the
> capability CAMEL-23944 is establishing for camel-langchain4j-agent.
> * {{hallucinatedToolNameStrategy=failExchange|repromptModel}} (default
> {{failExchange}}, current documented behavior). {{repromptModel}} feeds back
> e.g. {{"Error: tool 'x' does not exist. Available tools: ..."}}.
> The loop already has {{maxToolIterations}} to bound the extra round trips
> these strategies can cause. Documentation ({{openai-mcp.adoc}} error handling
> table) must be updated accordingly. Cross-references: CAMEL-23943/CAMEL-23944
> (langchain4j analogues), CAMEL-23393 (token budget also bounds reprompting),
> CAMEL-23382 (step 6 alignment).
> _This issue was drafted by Claude Code on behalf of Federico Mariani_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
