[
https://issues.apache.org/jira/browse/CAMEL-25639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18127084#comment-18127084
]
Claus Ibsen edited comment on CAMEL-25639 at 10/14/26 12:21 PM:
----------------------------------------------------------------
Thanks for the review. Folding the subsets and copies makes sense. One request
from the local-model runs we do against camel-jbang-mcp (an authoring benchmark
with qwen3.6 35B-A3B and Qwen3.8 27B, about 1,200-1,300 tool calls per night):
Please keep dedicated tools for the authoring and runtime loop: {{camel_run}},
{{camel_control}}, {{camel_get_log}}, {{camel_get_errors}},
{{camel_get_files}}, {{camel_write_file}}, {{camel_edit_file}},
{{camel_validate_source}} and {{camel_eval_expression}}. They made 63-72% of
all calls in our last series, and the catalog tools most of the rest. A model
picks a tool by its name and description, and one clear verb per tool with a
few typed arguments is what small local models get right. A generic dispatch
tool, something like {{camel_runtime(action, arg1, arg2)}}, moves that choice
into an action string and loosely typed arguments.
On the argument normalisation: our harness, and likely other clients and
prompts, call {{camel_run}} (directory), {{camel_control}} (name, action),
{{camel_get_log}} (name, limit), {{camel_get_errors}} (name) and
{{camel_runtime_tool_groups}} (nameOrPid) directly. If the process selector
becomes one name, please accept the old one too for a release.
_Claude Code on behalf of davsclaus_
was (Author: davsclaus):
Thanks for the review. Folding the subsets and copies makes sense. One request
from the local-model runs we do against camel-jbang-mcp (an authoring benchmark
with qwen3.6 35B-A3B and Qwen3.8 27B, about 1,200-1,300 tool calls per night):
Please keep dedicated tools for the authoring and runtime loop: {{camel_run}},
{{camel_control}}, {{camel_get_log}}, {{camel_get_errors}},
{{camel_get_files}}, {{camel_write_file}}, {{camel_edit_file}},
{{camel_validate_source}} and {{camel_eval_expression}}. They made 63-72% of
all calls in our last series, and the catalog tools most of the rest. A model
picks a tool by its name and description, and one clear verb per tool with a
few typed arguments is what small local models get right. A generic dispatch
tool, something like {{camel_runtime(action, arg1, arg2)}}, moves that choice
into an action string and loosely typed arguments.
On the argument normalisation: our harness, and likely other clients and
prompts, call {{camel_run {directory}}}, {{camel_control {name, action}}},
{{camel_get_log {name, limit}}}, {{camel_get_errors {name}}} and
{{camel_runtime_tool_groups {nameOrPid}}} directly. If the process selector
becomes one name, please accept the old one too for a release.
_Claude Code on behalf of davsclaus_
> camel-jbang-mcp - consolidate the tool surface (86 to about 66 tools) and
> normalise arguments, hints and output bounds
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25639
> URL: https://issues.apache.org/jira/browse/CAMEL-25639
> Project: Camel
> Issue Type: Improvement
> Components: camel-jbang-mcp
> Reporter: Federico Mariani
> Priority: Major
> Attachments: camel-jbang-mcp-review-2026-10.md
>
>
> camel-jbang-mcp lists 86 tools on main. In an evaluation of coding agents
> (Claude Code and IBM Bob) on legacy-to-Camel migration tasks with 4.22.1, 647
> calls used 22 of the 73 tools then exposed; the catalog tools, the
> validators, {{camel_component_properties}} and
> {{camel_configuration_validate}} were 98% of the calls, and several of the
> tools that were called answered with something a model could not use
> ({{camel_component_properties}} 15-124 K for one option name,
> {{camel_properties_translate}} main-to-main no-ops,
> {{camel_catalog_languages}} unfiltered 12-14 K). A code review of all 86
> tools (October 2026) then found that about twenty are subsets, composites or
> copies of another tool, several with descriptions that promise more than the
> code does. Dropping and folding them removes wrong answers from the path the
> agents take, makes the surface easier for a model to pick from, and makes the
> profiles of CAMEL-25537 smaller to define.
> The first group is the one the evaluation exercised; the second was never
> called in any session and is a cleanup.
> h3. First: tools on the agents' path
> || Tool || Into || Why ||
> | {{camel_component_properties}} | {{configKey}} on the component options of
> {{camel_catalog_doc}} | Same {{ComponentModel}}; its endpoint-options claim
> is false (CAMEL-25541). |
> | {{camel_catalog_docs}} | drop, or a {{kind=page}} in {{find}} | A page
> index whose names {{camel_catalog_doc}} resolves only for the four artifact
> kinds ({{CatalogDocs.java:227, 246, 270, 444}}). |
> | {{camel_configuration_validate}} | {{camel_validate_source}} on a
> {{.properties}} file | {{validate_source}} already runs the same catalog
> validation per line ({{PropertiesChecks.java:137}}) plus nested-group hints,
> closest-name suggestions, line numbers and {{fixes}}; add
> {{runtime}}/{{platformBom}} there. CAMEL-25538 stays a catalog fix. |
> h3. Second: never called in any evaluation session
> || Tool || Into || Why ||
> | {{camel_runtime_receive}} | drop | Leaks a consumer (see the code-review
> umbrella issue). |
> | {{camel_catalog_kamelets}}, {{camel_catalog_kamelet_doc}} |
> {{camel_catalog_doc}} / {{camel_catalog_find}} {{kind=kamelet}} (CAMEL-25379)
> | Same catalog jar; {{catalog_doc}} already adds the YAML shape, suggestions
> and project Kamelets. Add {{type}}, {{supportLevel}}, {{dependencies}} to the
> kamelet answer and let {{find}} take an empty term with a {{type}} filter. |
> | {{camel_migration_compatibility}} | {{camel_migration_recipes}} with the
> Java check from {{camelReleases()}} | Static and wrong data, unused required
> argument (see the migration data issue). |
> | {{camel_migration_analyze}} | {{camel_dependency_check}} {{projectInfo}} |
> Subset ({{DependencyCheckTools.java:87-88}});
> {{camel_migration_wildfly_karaf}} takes {{sourceRuntime}} instead of a POM. |
> | {{camel_openapi_mock_guidance}} | {{camel_openapi_scaffold}} output when
> {{missingOperation=mock}} | Static text plus a duplicate per-operation walk. |
> | {{camel_runtime_route_analysis}}, {{camel_runtime_eip_stats}},
> {{camel_runtime_config_drift}} | drop | Composites of two unbounded dumps
> each with six static hint strings and no analysis, aggregation or comparison
> ({{ToolRegistry.java:681-774}}), despite descriptions promising all three. |
> | {{camel_runtime_ai_trace}} | {{filter=ai}} on {{camel_runtime_history}} /
> {{top}} | Only true composite; sees one exchange. |
> | {{camel_route_context}} | {{camel_project_overview}} with a {{content}}
> argument | Its EIP extraction is a substring match over all model names
> ({{ExplainTools.java:106-122}}): every YAML route "uses" to, from, step,
> route; {{kafka:topic}} matches {{to}}. Use {{RouteNodes}}/{{RouteModel}}. |
> | {{camel_route_harden_context}} | {{camel_security_scan}} behind
> {{includeContext}} | Same regexes, coarser ({{HardenTools.java:181-243}} vs
> {{SecurityScanTools.java:202-326}}); the prompts call only harden, never the
> line-level scanner. Keep the catalog component context and the CVE match. |
> | {{camel_runtime_heap_histogram}}, {{heap_dump}}, {{memory_leak}} | one
> {{camel_runtime_heap view=histogram|dump|jfr}} | |
> | {{camel_runtime_trace}} | split the read ({{dump}}) from
> {{enable}}/{{disable}} | Today the whole tool is {{readOnlyHint=false}}, so
> read-only clients cannot dump traces. |
> {{ExampleTools}} becomes a thin wrapper of the registry (as
> {{AuthoringTools}} is): duplicated code with divergent not-found behaviour,
> CAMEL-25535.
> h3. Arguments
> * Process selector: {{nameOrPid}} on the runtime tools, {{name}} on
> {{camel_control}}/{{get_log}}/{{get_errors}}/{{eval_expression}}; required on
> {{ai_trace}} only.
> * Booleans typed {{String}} in {{heap_dump}}, {{memory_leak}}, {{sql}};
> {{limit}} {{Integer}} in browse/get_log but {{String}} in spans ({{limit}})
> and sql ({{maxRows}}).
> * Route selector {{routeId}} vs {{routeFilter}}; {{headers}} as {{key=value}}
> lines in {{send}} but JSON or {{name: value}} in {{http_request}}.
> * Three camelVersion descriptions ({{VERSION_DESC}},
> {{ToolArgDocs.CAMEL_VERSION}}, {{ToolArgDocs.VERSION_QUERY}} which asks for a
> Quarkus platform version); {{kind}} vocabularies differ between doc, find and
> sample; {{limit <= 0}} throws in docs, defaults in examples, is used raw in
> kamelets.
> * Four tools take a POM as inline content; none takes a {{directory}} like
> {{camel_project_overview}} does.
> h3. Output bounds the consoles support and the registry never passes
> {{route_source}} ({{filter=*}} is every file in full),
> {{route_dump}}/{{route_structure}} ({{limit}}, {{brief}}), {{thread_dump}}
> (full stacks of every thread), {{trace}} dump ({{limit}}, {{bodyMaxChars}}),
> {{history}} ({{summary}} exists in the registry, {{ToolRegistry.java:220}},
> not in the MCP wrapper), {{browse}} ({{bodyMaxChars}}),
> {{properties}}/{{endpoints}}/{{services}} (no filter).
> h3. Hints
> {{send}} destructive in MCP, not in the registry; {{route_control}}
> destructive for start/resume; {{sql}} destructive blocks read-write clients
> from SELECT (a SELECT-only check would allow it); {{heap_dump}} name
> unvalidated; {{write_file}} overwrites whole files as non-destructive.
> h3. Also worth doing on the way
> Hand-written tables with dead schemes: {{SecurityData}} lists websocket,
> rabbitmq, redis, ldaps, aws2-secrets-manager, cassandraql (not in the
> catalog; a component with an option marked {{secret}} or {{security=}} is
> derivable); {{TestInfraData}} maps 16 of 76 test-infra modules and sends
> sftp/ftps to plain FTP; {{DependencyData}} core-transitive list is only right
> for the {{camel-core}} aggregate. Regex heuristics with concrete false
> positives: {{security_scan}} flags {{to:}} producers and YAML {{- rest:}}
> blocks as consumers without a header filter, any {{sql:}} without {{:#}}, a
> log line with {{file:}} and {{${}}; {{route_cost_estimate}} prices Bedrock as
> Claude Sonnet regardless of {{modelId}}; {{route_test_scaffold}} generates a
> new {{from(first)}} to mock route so nothing of the real route is tested;
> {{camel_render_route_diagram}} says "parsed without running it" but boots a
> transient {{Run}}. Doc rows to refresh: DOC:349 ("only a plain file name"),
> DOC:493 (example levels), DOC:601 ("17 exceptions", 19), {{camel_control}}
> omits {{reload}}, {{tool_groups}} omits http/semantic,
> {{camel_migrate_project}} says pomContent required.
> Blocks CAMEL-25537: define the profiles over the reduced surface.
> h3. Added 2026-10-14: name the tools for the hosts that pick by name
> How the agent hosts load the server (checked in their sources and docs):
> || Host || Tool schemas at session start || How the model reaches a tool ||
> Server {{instructions}} ||
> | Claude Code 2.1.29x | deferred: tool names only ({{ENABLE_TOOL_SEARCH}}
> unset = always defer; {{auto}} = defer above 10% of the context) | ToolSearch
> {{select:<name>}}: 113 such calls in the evaluation, every one by name |
> loaded at session start, up to 4096 characters |
> | Codex CLI (main, 2026-10-14) | deferred: tool search is on by default
> ({{core/tests/suite/search_tool.rs}}, {{mcp_tool_exposure.rs:84}}) |
> tool_search | read from {{initialize}}, used as server-wide guidance; the
> first 512 characters matter most; {{enabled_tools}}/{{disabled_tools}} per
> server on the client side |
> | OpenCode (main, 2026-10-14) | code mode by default: tools are not declared,
> the prompt carries one catalog line per tool | an {{execute}} script calling
> {{tools["<server>"]}} | full text in an {{<mcp_instructions>}} block, re-sent
> when it changes |
> | Pi (main, 2026-10-14) | {{codemode}} by default (scripts use
> {{searchTools()}}), {{deferred}} via tool_search, {{direct}} per tool |
> scripts or tool_search | only the FIRST LINE reaches the system prompt; the
> full text through {{describeNamespace()}} in scripts |
> | Bob Shell 2.0.5 | all schemas up front (995 evaluation calls, no search
> step) | direct | not documented |
> On Claude Code, Codex and Pi the model chooses a tool by its NAME before it
> sees any description (113 ToolSearch calls in the evaluation, all
> {{select:<name>}}); on OpenCode and Pi code mode it sees one catalog line per
> tool. So for the consolidation: a name must state the question it answers,
> the first sentence of a description is the catalog line, and the unified
> {{camel_catalog_doc}} (components, EIPs, languages, data formats, Kamelets,
> beans, Java API, config groups under one name) should either keep a name per
> question or be backed by the instructions map of CAMEL-25646. In the
> evaluation {{camel_catalog_component_doc}} was selected 53 times by name;
> nothing in the list is named for the HTTP port or for what to declare for a
> JDBC driver, and those two questions were the largest waste (jar archaeology
> in plain and MCP arms, 44 knowledge searches).
> _Claude Code on behalf of Croway_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
