[
https://issues.apache.org/jira/browse/CAMEL-23952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Federico Mariani updated CAMEL-23952:
-------------------------------------
Description:
LangChain4j's {{AiServices.executeToolsConcurrently()}} /
{{executeToolsConcurrently(Executor)}} runs all tool calls of one LLM round
trip in parallel. For agents wired to several independent tools (MCP servers,
Camel route tools, custom tools) this cuts round-trip latency from the *sum* of
the tool durations to the *max* — e.g. an LLM requesting weather + user profile
+ inventory in one turn.
The option cannot be exposed today because the Camel {{ToolExecutor}} in
{{LangChain4jAgentProducer.createCamelToolProvider}} is not safe for concurrent
execution: it mutates the *shared live exchange* (writes tool-argument headers,
{{CamelToolName}}, and the body on the exchange being processed, then runs the
tool route on it). Two tools executing concurrently would corrupt each other's
state.
*Proposal* (two steps, second depends on first):
# Make the Camel tool executor exchange-safe: run each tool invocation on its
own {{ExchangeHelper.createCopy}} of a baseline exchange (the pattern
camel-langchain4j-tools already uses), never on the live exchange. This
overlaps with the fix for "ToolExecutionErrorHandler never invoked for Camel
route tools" — the exception-propagation bug has the same root cause. Define
what flows back to the main exchange: with sequential execution the current
header side-effects can be preserved via {{copyResults}}; with concurrent
execution header merge order is ambiguous, so document that tool-route header
side-effects are not propagated (or last-wins) in concurrent mode.
# Expose the option on {{AgentConfiguration}}:
{{withExecuteToolsConcurrently()}} and
{{withExecuteToolsConcurrently(Executor)}}, wired null-guarded in
{{AbstractAgent.configureBuilder()}}. When no explicit {{Executor}} is given,
prefer a thread pool from the CamelContext's {{ExecutorServiceManager}}
(managed thread pool profile, proper lifecycle and naming) over LangChain4j's
internal default.
Related: CAMEL-23944 (prerequisite), CAMEL-23928.
_This issue was drafted by Claude Code on behalf of Federico Mariani._
was:
LangChain4j's {{AiServices.executeToolsConcurrently()}} /
{{executeToolsConcurrently(Executor)}} runs all tool calls of one LLM round
trip in parallel. For agents wired to several independent tools (MCP servers,
Camel route tools, custom tools) this cuts round-trip latency from the *sum* of
the tool durations to the *max* — e.g. an LLM requesting weather + user profile
+ inventory in one turn.
The option cannot be exposed today because the Camel {{ToolExecutor}} in
{{LangChain4jAgentProducer.createCamelToolProvider}} is not safe for concurrent
execution: it mutates the *shared live exchange* (writes tool-argument headers,
{{CamelToolName}}, and the body on the exchange being processed, then runs the
tool route on it). Two tools executing concurrently would corrupt each other's
state.
*Proposal* (two steps, second depends on first):
# Make the Camel tool executor exchange-safe: run each tool invocation on its
own {{ExchangeHelper.createCopy}} of a baseline exchange (the pattern
camel-langchain4j-tools already uses), never on the live exchange. This
overlaps with the fix for "ToolExecutionErrorHandler never invoked for Camel
route tools" — the exception-propagation bug has the same root cause. Define
what flows back to the main exchange: with sequential execution the current
header side-effects can be preserved via {{copyResults}}; with concurrent
execution header merge order is ambiguous, so document that tool-route header
side-effects are not propagated (or last-wins) in concurrent mode.
# Expose the option on {{AgentConfiguration}}:
{{withExecuteToolsConcurrently()}} and
{{withExecuteToolsConcurrently(Executor)}}, wired null-guarded in
{{AbstractAgent.configureBuilder()}}. When no explicit {{Executor}} is given,
prefer a thread pool from the CamelContext's {{ExecutorServiceManager}}
(managed thread pool profile, proper lifecycle and naming) over LangChain4j's
internal default.
Related: "camel-langchain4j-agent: ToolExecutionErrorHandler and
compensateOnToolErrors never invoked for Camel route tools" (prerequisite),
CAMEL-23928.
_This issue was drafted by Claude Code on behalf of Federico Mariani._
> camel-langchain4j-agent: support executeToolsConcurrently by making the Camel
> tool executor exchange-safe
> ---------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-23952
> URL: https://issues.apache.org/jira/browse/CAMEL-23952
> Project: Camel
> Issue Type: Improvement
> Components: camel-langchain4j-agent
> Reporter: Federico Mariani
> Priority: Major
>
> LangChain4j's {{AiServices.executeToolsConcurrently()}} /
> {{executeToolsConcurrently(Executor)}} runs all tool calls of one LLM round
> trip in parallel. For agents wired to several independent tools (MCP servers,
> Camel route tools, custom tools) this cuts round-trip latency from the *sum*
> of the tool durations to the *max* — e.g. an LLM requesting weather + user
> profile + inventory in one turn.
> The option cannot be exposed today because the Camel {{ToolExecutor}} in
> {{LangChain4jAgentProducer.createCamelToolProvider}} is not safe for
> concurrent execution: it mutates the *shared live exchange* (writes
> tool-argument headers, {{CamelToolName}}, and the body on the exchange being
> processed, then runs the tool route on it). Two tools executing concurrently
> would corrupt each other's state.
> *Proposal* (two steps, second depends on first):
> # Make the Camel tool executor exchange-safe: run each tool invocation on its
> own {{ExchangeHelper.createCopy}} of a baseline exchange (the pattern
> camel-langchain4j-tools already uses), never on the live exchange. This
> overlaps with the fix for "ToolExecutionErrorHandler never invoked for Camel
> route tools" — the exception-propagation bug has the same root cause. Define
> what flows back to the main exchange: with sequential execution the current
> header side-effects can be preserved via {{copyResults}}; with concurrent
> execution header merge order is ambiguous, so document that tool-route header
> side-effects are not propagated (or last-wins) in concurrent mode.
> # Expose the option on {{AgentConfiguration}}:
> {{withExecuteToolsConcurrently()}} and
> {{withExecuteToolsConcurrently(Executor)}}, wired null-guarded in
> {{AbstractAgent.configureBuilder()}}. When no explicit {{Executor}} is given,
> prefer a thread pool from the CamelContext's {{ExecutorServiceManager}}
> (managed thread pool profile, proper lifecycle and naming) over LangChain4j's
> internal default.
> Related: CAMEL-23944 (prerequisite), CAMEL-23928.
> _This issue was drafted by Claude Code on behalf of Federico Mariani._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
