[
https://issues.apache.org/jira/browse/CAMEL-23861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18092935#comment-18092935
]
Claus Ibsen commented on CAMEL-23861:
-------------------------------------
h3. Two separate AI token-producing paths
There are two distinct sources of AI/LLM token usage in Camel:
*1. Camel application routes (langchain4j components)* — token data lands on
exchange headers ({{CamelLangChain4j*TokenCount}}). This ticket (CAMEL-23861)
is about propagating this into the OTel/Micrometer observability layer so
production AI usage in routes can be monitored via Grafana, Jaeger, Prometheus,
etc.
*2. {{camel ask}} CLI / TUI AI panel* — uses {{LlmClient}} directly (not
langchain4j, not Camel routes). Talks raw HTTP to OpenAI/Anthropic/Ollama APIs.
Token tracking is in-memory only inside {{LlmClient.TokenUsage}} and
{{AiPanel.sessionTotalTokens}}. This is the *developer's own* AI usage,
separate from the running Camel application.
h3. TUI AI Usage tab approach
Inspired by [boot-ui's AI Usage
panel|https://github.com/jdubois/boot-ui/blob/main/docs/FEATURES.md#ai-usage]
which reads OpenTelemetry GenAI spans from an in-memory telemetry store.
The plan is to build a TUI tab for AI/LLM usage visualization. The {{camel
ask}} token data (already tracked in AiPanel) is a natural starting point to
build and test the UI — it gives developers direct visibility into how much
their AI prompts cost in tokens. The app-side observability from langchain4j
routes (this ticket) comes later as a second data source for the same tab,
showing production AI usage in the running Camel application.
> camel-langchain4j - Add OpenTelemetry and Micrometer observability for AI/LLM
> usage
> -----------------------------------------------------------------------------------
>
> Key: CAMEL-23861
> URL: https://issues.apache.org/jira/browse/CAMEL-23861
> Project: Camel
> Issue Type: Improvement
> Components: camel-langchain4j-agent, camel-langchain4j-chat,
> camel-langchain4j-embeddings, camel-langchain4j-tools
> Reporter: Claus Ibsen
> Priority: Major
> Labels: ai, observability
> Fix For: 4.22.0
>
>
> The camel-langchain4j components currently have no observability integration
> for AI/LLM-specific metrics and tracing. Token usage, model name, finish
> reason, and latency are not captured.
> Spring AI exposes this data through the Micrometer Observation API (which
> produces both metrics and traces) following the OpenTelemetry Semantic
> Conventions for GenAI (https://opentelemetry.io/docs/specs/semconv/gen-ai/).
> This enables dashboards showing token consumption, costs, model distribution,
> and conversation tracking.
> Camel should follow the same approach. When camel-micrometer or
> camel-opentelemetry2 is active, the langchain4j producers should capture:
> *OpenTelemetry span attributes (GenAI semantic conventions):*
> - gen_ai.system (e.g. openai, anthropic)
> - gen_ai.request.model
> - gen_ai.response.model
> - gen_ai.usage.input_tokens
> - gen_ai.usage.output_tokens
> - gen_ai.response.finish_reasons
> *Micrometer metrics:*
> - Token counters (input/output/total) with model tag
> - Request counters per model/operation
> - Latency timers/histograms
> - Error counters
> This would allow standard observability tooling (Grafana, Jaeger, Prometheus,
> etc.) and Camel TUI to visualize AI/LLM usage across routes.
> See also CAMEL-23860 which adds token data as exchange headers - this ticket
> is about propagating that data into the observability layer for monitoring
> and dashboards.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
