[
https://issues.apache.org/jira/browse/CAMEL-25618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luigi De Masi updated CAMEL-25618:
----------------------------------
Description:
h2. Motivation
Build on the semantic auditing delivered by CAMEL-25499 to give operators a
consistent way to inspect the expert's returned result and the route where the
evaluation was invoked, follow related evaluations, and observe changes in
results over time. Simplify semantic audit to automatically captured
evaluations, their execution lifecycle and runtime provenance.
A fixed-purpose injection classifier, an instruction-driven choice or score
expert, and a moderation or business-policy expert should be understandable
through the same vocabulary. Operators also need counts such as “how many
requests selected billing?”, “how many Boolean answers were true?” and “has the
urgency distribution changed?”, without deriving production metrics from
whichever Audit page happens to be open.
Deliver the complete scope below in [PR
#27791|https://github.com/apache/camel/pull/27791], organized into four
reviewable commits: explicit expert requirements, audit foundation and
presentation, shared monitoring and presentation, and playground/layout
controls. Diagnostics, monitoring, metrics and the supporting capabilities
remain part of this enhancement. Intended fix version: *4.24.0*.
h2. Shared model and ownership
*Drop route-action auditing and route-authored business metadata from the
target design.* Remove the separate route-decision concept from new capture,
configuration, the shared presentation model and the TUI/CLI/MCP workflows. Do
not retain it as an optional feature. This removes action declarations, action
vocabularies, application operation/target, policy/rule metadata, policy
reasons, action-specific filters/counters and decision-to-evidence panels.
Audit should expose what the expert returned and how the evaluation ran.
Ordinary route behavior stays the responsibility of the application and is not
represented as a semantic audit decision.
Keep *Result*, *Execution status*, *Assessment operation* and *Execution
reason* distinct. Preserve the expert's returned Boolean, choice, score or
classification, its captured meaning, expert instance/provider, definition
name, invocation origin and recorded route/exchange/breadcrumb context. A
Boolean true, classification label or high score never implies allow/block or
severity. Auditing requires no decision bean, route annotation,
business-metadata block or cross-reference beyond its ordinary configuration.
Original result interpretation remains captured automatically; it is separate
from the business annotations being dropped.
*Camel exposes information; experts and applications own its meaning.* Capture
and expose supplied interpretation, model/revision and optional task/version
identifiers as reported. Camel does not detect meaning changes, derive semantic
identities, require authors to version every question, decide whether two
populations mean the same thing, or automatically split series/reset baselines.
Meaning, versioning, grouping and comparison policies belong to the expert,
application or external monitoring system. Missing metadata is displayed as not
provided.
*The TUI is a visualization surface.* Its time windows, filters, comparisons
and display pause affect queries and presentation only. Collection, retention,
thresholds, active detector baselines and notification behavior are configured
outside the TUI. Existing audit configuration remains DSL-owned and frozen
while the service runs.
*Stores store information; calculation belongs to the producer of the
measurement.* Stores persist and retrieve audit records and already-computed
snapshots, and own retention, durability and recovery of stored information.
They do not calculate result statistics, case summaries, coverage or drift.
Camel can provide ordinary counting/formatting utilities and assemble explicit
audit links, but expert/application/monitoring producers own semantic grouping
and comparison policies. Camel exposes supplied measurements and their
availability. Memory and durable stores may offer different retention/recovery
guarantees.
Bulk retrieval has a separate responsibility boundary.
{{SemanticAuditReader.getAll(eventIds)}} retrieves records by ID without a
universal evidence-specific 100-ID cap. Console/API callers enforce
request-size and response budgets. Backend implementations manage their own
batch limits and may split retrieval into smaller storage queries. Preserve
legacy v1 evidence-record validation, existing query page limits and input
bounds. Support existing readers that still enforce the old cap through bounded
caller-side chunking during migration. Do not silently truncate requested IDs
or treat a lookup failure as a missing record. This requires runtime/SPI and
caller changes, with no new persisted field or breaking storage-schema change.
Keep responsibilities small and explicit:
||Part||Responsibility||
| Expert/adapter | Assess the submitted input and declare result meaning. |
| Route | Invoke the expert and continue normal application routing; no
separate audit decision declaration. |
| Audit runtime | Capture evaluation results, execution outcomes and available
runtime context automatically; associate evaluations, batch members and request
outcomes using their recorded identifiers. |
| Metric/monitoring producer | Own measurement semantics, grouping and
comparisons; supply measurements for exposure. The producer can be an expert,
application or external monitoring system. |
| Store | Persist and retrieve supplied records/state; implement retention and
recovery. |
| Console/JMX | Expose the shared records and computed snapshots through their
respective read interfaces. |
| TUI/CLI/MCP presentation | Format and present that data, with read-only
navigation, filters and visualization windows. |
These are responsibility boundaries, not a request for a new framework or
module for each row. Reuse existing Camel infrastructure, give each calculation
one owner, and avoid separate accumulators or monitoring logic in stores,
management interfaces and clients.
h2. Proposed scope
h3. 1. Consistent Audit, CLI and playground presentation
Introduce a shared result formatter for Boolean, choice, score and
classification, used by Audit, CLI and the playground with the appropriate
captured/submitted interpretation. Show exact fractional scores, empty label
sets, omitted results, keyed probabilities and confidence without rounding them
into policy judgments. Probability and confidence retain their own meanings.
Use bordered panels: context and visualization controls at the top, the
evaluation table in the centre, a selected-record inspector below it, and
persistent health and keyboard hints. Keep expert identity and typed result
visible. At 144x40, show Timestamp, Executing route, Evaluation,
Expert/provider, Result, Execution, Duration and Breadcrumb. The selected
evaluation's inspector presents returned result and captured meaning beside
execution/provenance details. At 100x30, retain Timestamp, Executing route,
Expert, Result and Execution in the table; move the definition name, provider
and identifiers into the single-column details, with an expanded inspector
available. At 88x24, use a compact list and a full-width scrollable inspector.
Request lifecycle records remain accessible through their invocation
relationship. Do not add route-action panels, selectors or alternate views. Use
text labels alongside colour and local timestamps with clear date/time-zone
information. The mockups illustrate this layout; they are not required to
understand the feature.
*Executing route is automatic runtime provenance.* Capture the route invoking
the evaluation from the Exchange's active route context when evaluation starts,
before expert work is dispatched. Camel already exposes this through
{{ExchangeHelper.getAtRouteId(exchange)}}. The currently stored {{routeId}},
obtained from {{exchange.getFromRouteId()}}, identifies the entry route;
preserve its meaning and existing filter behavior. Propose a separate bounded
optional {{executingRouteId}} field, subject to the compatibility rules below.
For an exchange entering {{http-entry}} and invoking an evaluation inside
{{prompt-screen}}, show Executing route {{prompt-screen}} and retain Entry
route {{http-entry}} in details. Do not reconstruct the executing route from an
exchange/breadcrumb identifier, today's route definitions or completion-time
context. No author-supplied metadata or new DSL option is required. Older
records show executing route as “not recorded”; calls without an Exchange show
“n/a.” If an Exchange has no active route context, leave that identity
unavailable rather than substituting the entry route. Missing executing-route
capture never prevents browsing existing history.
*Store both identifiers; keep the table concise.* {{exchangeId}} and
{{breadcrumbId}} already belong to the version-1 record. When both are
available, including a valid breadcrumb, retain both as separate fields in the
same audit record. Exchange ID identifies the individual Exchange; Breadcrumb
ID associates related exchanges. Auditing copies Camel's existing breadcrumb
and does not generate a replacement or substitute Exchange ID under that name.
Storage durability and retention remain backend responsibilities; viewing
history reads the stored identifiers without needing the live Exchange. This is
independent of raw-input capture and needs no new schema fields for either
identifier.
*Breadcrumb ID is the only correlation identifier shown as a table column.
Exchange ID is detail-only.* Every Audit/diagnostics detail view shows both
recorded identifiers with explicit labels and access to their full values; long
table values use visible ellipsis. At narrow widths, breadcrumb also moves into
details to keep Executing route and Result readable. An absent breadcrumb is
labelled “not recorded,” without claiming why it is missing. Missing
identifiers do not prevent audit inspection. Neither ID is a route location,
and a shared breadcrumb alone does not prove causal linkage or case
completeness.
Add visualization time presets, quoted exact filter values, selectors and
explicitly scoped refinements such as text, execution state and result
availability. Preserve the reader's exact-match contract. The existing
{{routeId}} filter remains an Entry route filter; filtering by Executing route
must use an explicitly supported query field or a clearly labelled loaded-page
refinement. Page-local results and counters must be labelled as such. Complete
window counts come from runtime-computed snapshots or a bounded runtime
calculation over sufficiently complete retained data; the store is not required
to calculate them. Stabilize live selection, report new arrivals and make
pause, older pages and expired cursors clear. Update CLI text, JSON, help and
documentation consistently.
h3. 2. Cases, relationships, health and export
Group related evaluations and request outcomes by a source-scoped breadcrumb,
with clearly labelled exchange, invocation and event fallbacks when necessary.
Show batch siblings and request-to-completion associations using captured
identifiers. Group membership is association, not proof of causation or
completeness.
Distinguish related records not yet loaded, unavailable records, omitted
results and incomplete retrieval scope. An evaluation is independently
inspectable; do not introduce missing-action or missing-business-metadata
warnings. Do not infer the reason an individual record is unavailable from a
global eviction counter.
Separate capture, reader, delivery and observed expert health. Show
per-instance collection state and execution outcomes, retained capacity,
eviction/drop counts and sink/observer failures. Keep errors visible at narrow
widths and distinguish disabled capture, no matches and a failed backend.
Support visible failure notifications and runtime-configured operational
notification hooks without giving the TUI control over them.
Provide bounded case export and structured MCP case summaries with retrieval
scope, relationship completeness and missing references. Summaries exclude
captured input. Explicit input-bearing details/exports identify untrusted data
and capture/redaction/omission state; apply the same boundary to generic
TUI-to-AI inspection paths.
h3. 3. Preserve historical meaning and support useful diagnostics
Capture the complete contract-declared original interpretation once in the
typed semantic event: relevant choice keys/descriptions, score levels, Boolean
labels and explicit no-match or probability-domain relationships where
available. Freeze effective meaning at invocation time; editing or deleting a
definition must not reinterpret that event or its retained history. There is no
producer-side interpretation truncation limit. Each retaining consumer enforces
its own payload, queue and storage budget. The audit recorder creates a bounded
projection before enqueueing, capability-gates additive fields for its
store/sinks, and reports explicit omission reasons when its budget or
capabilities prevent retaining interpretation. No separate
interpretation-capture opt-in is required. Raw-input capture retains its
existing separate opt-in; interpretation capture does not enable arbitrary
prompt, instruction or parameter capture. Older records remain readable, with
missing interpretation labelled “not recorded”; never reconstruct it from
today's definition.
Enable new persisted interpretation/finding fields and executing-route
provenance only when the configured store/sinks declare support. Unsupported
capture is explicitly visible through health/detail responses, without sending
unrecognized fields to older implementations. Lack of extension support does
not by itself reject the existing configuration or prevent startup.
Derive diagnostics from existing results: reported top-two probability gap,
alternatives and ties, explicitly designated no-match, and score modes where a
declared mapping supports them. Preserve the selected answer even when it
differs from the highest reported probability. Mark incomplete distributions
and unavailable interpretation; diagnostics are neither model reasoning nor
calibrated certainty. No extra inference is required.
For example, a choice result selecting billing with reported probabilities
billing=0.68, technical=0.29 and other=0.03 can display the reported top-two
gap of 0.39 and technical as the next alternative. An ordinary key named other
is not automatically a no-match answer. A score of 1.6 remains 1.6; captured
level descriptions may explain its position between levels, but cannot turn it
into a discrete score of 2 or a severity.
Allow explicit transfer of permitted captured input into a new playground
draft. Identify missing original parameters and changed execution context;
never claim exact replay. Browsing and draft creation do not invoke an expert.
Evaluation remains the playground's separate, explicit Run action.
h3. 4. Typed-result metrics and expert-owned MBeans
Provide shared utilities for bounded typed-result counters/distributions and
expose expert-owned measurements independently of audit retention and TUI
polling. Measurement producers supply their dimensions and any task/version
identifiers; Camel does not derive semantic identity or manage series versions:
* *Choice:* count the selected answer once per successful evaluation; include
proportions and denominators. Probability entries are not additional answers.
* *Boolean:* count the returned true/false answer. For TypeSafe yes/no
questions, use the adapter's final Boolean; do not apply another threshold. A
reported yes-probability histogram is a separate metric. Failures and
unavailable results are not false answers.
* *Score:* count, sum, minimum/maximum and bounded numeric histograms. Preserve
fractions; exact level counts require discrete-level semantics.
* *Classification:* per-label membership and empty-set counts. Multiple labels
may contribute to one evaluation; percentages need not sum to 100%.
Count completed assessment members, including batch members, rather than
provider calls. Separate provider attempts and semantic completions,
exclusions, unavailable results and collection loss. The metric owner defines
grouping and decides how question, criterion, model or policy changes affect
its series. Camel preserves the supplied dimensions and exposes their meaning;
it does not infer semantic continuity or automatically create a new series.
Expose shared health and expert-described metric snapshots through *JMX, TUI,
CLI and MCP*, using the same read-only source. JMX remains optional at
deployment. Experts own the measurements and supply their name, meaning, kind,
unit, values and bounded dimensions. They may own a facade and contribute
provider-specific metrics without duplicating collection. Use a small optional
description/snapshot contract through the console for generic presentation; do
not introspect arbitrary MBeans or add provider-specific TUI branches. Experts
without custom metrics need not implement this extension.
Use bounded tables/buckets, lifecycle registration and no MBean per
answer/event. Console and JMX consume the same snapshot; reads do not run
inference, scan all records or synchronously query a remote store. Include
interval/epoch, units, denominators and availability. Keep runtime cumulative
counters distinct from stored historical aggregates. Honor JMX-disabled
deployments; expose no reset, threshold, baseline or lifecycle mutations on the
new metrics facade.
h3. 5. Semantic Monitoring and AiEvent integration
Add a sibling Semantic Monitoring view for measurements, comparisons and
signals supplied by an expert/application/monitoring producer, with persisted
snapshots where configured. That producer owns comparison semantics; the store
saves/retrieves supplied state and Camel exposes it. Support visualization
against a supplied fixed reference by default, and a previous-period comparison
when provided or supported by that producer. Show reference/current bounds,
sample counts, exclusions/loss, reported revisions and source-provided
comparison state. Missing information remains not provided. Camel does not
infer comparability or manage semantic changes. A distribution change does not
establish accuracy loss or a security incident and never changes routing
behavior.
Operators may change visualization periods and inspect available comparisons.
Active detector policy remains owned by its producer outside the TUI; a
visualization query does not emit operational signals. Historical windows
depend on the information produced and retained. This enhancement exposes and
visualizes monitoring information without adding a Camel semantic-change
management engine. Latest snapshots refresh once per second. Sampled history
and persistence retain changed snapshots at most once per minute per series by
default, with an immediate first sample. A registered
SemanticMonitoringConfiguration bean exposes a runtime-changeable
historyInterval in seconds (1..86400). Explicit producer-published
snapshots/comparisons/signals bypass that sampling interval and are retained
immediately when changed. Unchanged samples do not create history or store
writes. Expose actual retained coverage and distinguish local capture times
from producer-reported windows; recovered history with unavailable local
capture times must say so. No durable monitoring backend is bundled.
Use the merged common AiEvent API from CAMEL-25408 as the shared occurrence
contract for Audit and Monitoring. Add component-owned typed Semantic events
for evaluation starts/completions and request outcomes, carrying captured
results, complete frozen interpretation, execution state and provenance.
Route-bound evaluation notifications implement the existing
{{AiEvent.ExchangeAiEvent}} contract and expose the live Exchange to standard
exchange-aware listeners. Detached direct calls and request outcomes use
exchange-free events; never create a synthetic Exchange. No new core event
hierarchy is needed. Keep expert identity and adapter provider in the typed
payload and use a stable publishing-family label. AiEvent is the specialized
Camel event contract/envelope that producers implement and consumers recognize.
Assessment fields deliberately belong to the Semantic specialization; the
common root remains generic. EventHelper publishes notifications and
EventNotifier defines the consumer contract. AiEvent itself does not define a
wire transport, queue or persistence format.
Construct and freeze the occurrence data once. Feed the typed event directly to
the existing audit recorder, which applies capture policy and its own retention
budget, attaches permitted captured input privately and sends the frozen record
through its bounded sink queue. Publish that same event through
{{EventHelper.notifyAiEvent}} for expert/monitoring consumers. The typed data,
JSON representation and safe string summary exclude raw input, but a
route-bound event deliberately exposes its live Exchange (including body and
headers) through {{getExchange()}}/{{getSource()}}. This is an in-process
listener contract, not a promise that the whole event is input-free. Listeners
must keep callbacks short, avoid payload dumps, and freeze/project any data
they retain; audit stores, queues and monitoring snapshots never retain the
live Exchange or raw event. Audit capture remains independent of
management-listener filters. Audit records remain the persisted history format.
Result-metric producers consume typed completion events once per assessment
member. They may publish already-computed metric, comparison and signal
snapshots as typed AiEvents; Camel exposes those supplied values through the
existing read-only snapshot surfaces. Distinguish starts, completions, request
outcomes and producer outputs so counters cannot double-count or consume their
own output as another evaluation. Producers retain ownership of calculations,
grouping, comparability and notification policy. Stores persist/retrieve
supplied information, and TUI/CLI/MCP display it.
Use the merged helper's notifier lifecycle, filtering and failure isolation,
with short callbacks and bounded consumer processing. Expose measured
loss/freshness; event dispatch is synchronous, best effort and supplies no
durable replay. Monitoring consumers and event production work with audit
retention disabled and no TUI attached. Preserve existing SemanticObserver
compatibility and its parent-context/detached tracing hooks; AiEvent does not
replace those guarantees. This integration requires runtime/component work, not
a breaking persisted-schema change or an OpenTelemetry dependency in
camel-semantic. Existing tool/loop events may support their own activity
metrics, but do not fabricate semantic results or route actions from them.
h3. 6. Expert-reported findings
Support bounded expert-authored findings with their supplied meaning and
severity provenance. Missing severity means not reported; never derive it from
a Boolean, score or confidence. Findings describe the expert's assessment and
remain supplementary to its returned typed result.
Drop route-authored policy priority, policy/rule coverage and
enforcement-evidence views that depended on the removed route-decision
metadata. Business-policy experts continue to report ordinary Boolean, choice,
score or classification results and expert-owned findings through the same
generic presentation.
h2. Compatibility and implementation areas
Prefer *(a) presentation changes* and *(b) optional, versioned console
enrichment* for relationships, aggregates and capabilities. Agree *(c) bounded
additive record fields* only where historical interpretation, findings or
executing-route provenance need persistence. Both identifiers already exist:
showing Breadcrumb ID in the table and both IDs in details is *(a)*. Showing
the existing entry route is also *(a)*; capturing the separate executing route
is runtime work with a proposed *(c)* field, requiring maintainer agreement.
Keep {{routeId}} and its query semantics unchanged. Existing v1 allowlists
require capability-gated emission: keep older records readable and existing
store/sink configurations usable, and visibly report unsupported capture. The
declaration/negotiation mechanism still requires API design. *No (d) breaking
record-schema change is proposed.* Runtime collectors, store capabilities and
management projections are additional work, not presentation-only changes.
*Migration of the dropped preview feature:* remove {{SemanticAuditDecision}}
and its route-decision recording/configuration path outright, with
documentation and examples using automatic evaluation capture. Semantic and
expert APIs are preview APIs and may change without a deprecation period;
document these removals in the 4.24 upgrade guide. Preserve the ability to
deserialize and export already-persisted v1 decision records as legacy data. Do
not delete stored history, reinterpret it as an evaluation, or remove accepted
fields from the v1 storage schema. Legacy compatibility does not restore route
actions as an optional workflow. Do not silently turn old recording calls into
no-ops.
*Explicit experts:* require an explicit registry expert on each evaluation.
Remove language default-expert configuration, sole-expert fallback, block-level
expert inheritance and adapter service discovery. Providers remain available
through explicitly registered SemanticExpert/SemanticAdapter beans. Update
declarations, schemas, resolver/tooling metadata, documentation and the upgrade
guide consistently. These are agreed preview API/configuration changes, not
removals of the expert/adapter extension contracts themselves.
Build on SemanticAuditRecord, SemanticAuditQuery, SemanticAuditConsole,
SemanticAuditService, SemanticObserver, SemanticAuditView, SemanticResultView,
the semantic CLI and TUI MCP tools. Add optional capabilities with explicit
unsupported/partial states; do not impose full-store scans on the one-second
refresh or require every custom reader/expert to implement every extension.
Collection and notification failures must not change inference or routing
behavior.
The interpretation/finding/provenance compatibility behavior and
storage-versus-calculation boundary are specified above. Implementation review
must settle bounded field definitions and capability negotiation,
retrieval/persistence contracts, shared snapshot responses and supplied
metric/identity metadata. There is no Camel-owned semantic identity or
meaning-change detection contract. The store contract contains no requirement
to calculate semantic aggregates or comparisons. Exact SPI signatures, field
names and management naming follow implementation review.
h2. Acceptance criteria
# Boolean, choice, fractional score, empty classification, omitted result,
failed execution and business-policy assessment fixtures retain distinct
meanings across TUI, CLI and MCP. Shared result formatting agrees with the
playground.
# Typed events carry complete frozen interpretation without producer-side
truncation. Audited evaluations automatically retain bounded projections on
compatible backends, without a separate opt-in or enabling raw-input capture;
each retaining consumer owns and enforces its budgets. Definition
edits/deletion do not change captured historical labels. Older records remain
readable with missing interpretation labelled “not recorded.” New fields are
emitted only with declared store/sink support; unsupported capture is visible
and does not prevent startup. Capture bounds and mixed-version behavior are
documented and tested.
# Route-bound and direct assessments, unavailable related records, absent
breadcrumbs, partial batches and cancellation-followed-by-completion are
navigable without fabricated results or completeness claims. Evaluation
auditing works without a decision bean, route annotation or business-metadata
configuration. Cases associate evaluations and lifecycle records through their
captured identifiers.
# The framed layouts work at 144x40, 100x30 and 88x24. Selection, input-loading
states, mixed dates, quoted filters, narrow-width errors and keyboard/help
parity are verified.
# Summary/list/metrics surfaces remain input-free. Explicit input detail, case
export and draft handoff preserve the trust boundary and never trigger
automatic inference.
# Known fixture populations produce correct choice, Boolean, numeric and
classification measurements, with batch members counted once and omissions/loss
visible. Supplied dimensions/revisions survive capture and exposure unchanged.
Camel neither infers semantic changes nor automatically splits, merges or
resets series; missing versions do not prevent ordinary audit/metric exposure.
# An expert-defined metric is visible through JMX, TUI, CLI and MCP with
matching supplied name, meaning, unit and values. Metrics are read-only,
cardinality is bounded, lifecycle cleanup works, and disabled JMX leaves
console-based functionality available. No surface independently redefines the
metric or duplicates its collection.
# Monitoring displays supplied fixed-reference and previous-period comparisons,
sample counts and source-reported unavailable/incomparable states. Stores
persist/retrieve supplied information; console/JMX expose the same snapshots.
Memory/durable recovery behavior and absent metadata are explicit. Camel does
not infer changes in meaning, impose a versioning scheme or manage comparison
policy. TUI visualization changes do not alter collection, detector state or
notifications.
# Typed Semantic AiEvents feed audit recording and expert/monitoring consumers
for route-bound and direct evaluations, with audit-disabled monitoring and no
TUI required. Each completed batch member is counted once; starts, request
outcomes and supplied monitoring outputs are distinct. Route-bound
notifications expose the live Exchange through ExchangeAiEvent; detached
notifications do not fabricate one. The frozen data/JSON and summary surfaces
exclude raw input, while opt-in audit input capture remains independent of
management-listener filters; history remains readable without an event
consumer. Verify listener failure/queue-loss behavior, supplied snapshot
exposure and compatibility with existing observer tracing hooks.
# Expert-reported findings retain their author, meaning and supplied severity
scale; absent severity is explicit. Documentation and examples cover a fixed
classifier, an instruction-driven expert and provider-independent
classification/business-policy data without route-authored policy metadata or
coverage claims.
# A nested-route fixture entering route A and invoking an evaluation in route B
records Entry route A and Executing route B at invocation time, without
additional author configuration. Completion, route renaming/deletion and
historical browsing do not change those recorded identities. Existing
{{routeId}} filters keep their entry-route meaning. Direct calls, missing
active route context, old records and unsupported capture are represented
without guessing a location or preventing normal audit operation.
# A record with both valid identifiers retains {{exchangeId}} and
{{breadcrumbId}} together through storage and retrieval. Breadcrumb ID is the
only correlation column in the wide table; Exchange ID is never a table column.
Detail boxes expose both full recorded values. At narrow widths both IDs remain
accessible in details while Executing route and Result stay readable. Absent
breadcrumbs are handled without generating or relabelling identifiers.
Summary/list responses remain input-free.
# Route-action auditing is dropped from the target capture/configuration model
and TUI/CLI/MCP presentation, including its metadata, filters, counters, panels
and policy-coverage dependencies. Migration removes the preview recording entry
point outright and replaces its examples with automatic evaluation auditing;
the 4.24 upgrade guide documents the removal. Already-persisted v1 decision
records remain deserializable/exportable as legacy data without restoring the
removed feature or breaking the storage schema.
# Bulk retrieval accepts a permitted request exceeding 100 IDs without an
evidence-specific SPI limit. Caller request/response budgets are enforced,
backend chunking returns all available requested records, and existing readers
with the old cap remain usable during migration. Missing records remain
distinct from retrieval failures. Legacy v1 evidence-record validation, query
page limits and input bounds remain unchanged.
# Every evaluation names its expert registry bean explicitly. Remove default
experts, sole-expert fallback, block-level experts and adapter service
discovery; migrate Java/XML/YAML declarations, validator schemas, tooling,
providers, examples and upgrade guidance. Missing expert references fail
clearly. This requirement does not remove the SemanticExpert or SemanticAdapter
extension contracts.
# Monitoring latest data refreshes every second, while changed sampled
history/store writes default to one minute per series with runtime
configuration. Explicit publications are retained immediately when changed.
Tests cover notifier filtering, audit-disabled event eligibility, restart
resets, retention cadence, runtime cadence changes and actual retained coverage.
h2. Related work and visual references
* [CAMEL-25499: semantic audit
foundation|https://issues.apache.org/jira/browse/CAMEL-25499] — follow-up to
the delivered implementation.
* [CAMEL-25357: semantic dev
console|https://issues.apache.org/jira/browse/CAMEL-25357] — overlapping
foundational proposal; coordinate scope and reuse the existing console/recorder.
* [CAMEL-25408: common AiEvent
API|https://issues.apache.org/jira/browse/CAMEL-25408] and [PR
#27714|https://github.com/apache/camel/pull/27714] — Resolved/Fixed and merged,
verified 2026-10-13; the common API is available on main for 4.24.0. Build the
Audit/Monitoring event integration on this foundation; no pending upstream
hierarchy/dispatch request remains.
* [CAMEL-23369: GenAI
observability|https://issues.apache.org/jira/browse/CAMEL-23369] — coordinate
common telemetry integration; this issue adds semantic result/case/monitoring
meaning.
* [Original framed layout
reference|https://issues.apache.org/jira/secure/attachment/13085107/13085107_camel-semantic-audit-tui.png]
— layout inspiration only; the proposed mockups use the information model
described here.
The attached mockups illustrate Audit, diagnostics and Monitoring at 144x40 and
100x30. They contain synthetic examples, not implemented UI screenshots. At
88x24, use a compact list and full-width scrollable inspector.
_AI-assisted content prepared by Codex on behalf of Luigi De Masi
(@luigidemasi)._
was:
h2. Motivation
Build on the semantic auditing delivered by CAMEL-25499 to give operators a
consistent way to inspect the expert's returned result and the route where the
evaluation was invoked, follow related evaluations, and observe changes in
results over time. Simplify semantic audit to automatically captured
evaluations, their execution lifecycle and runtime provenance.
A fixed-purpose injection classifier, an instruction-driven choice or score
expert, and a moderation or business-policy expert should be understandable
through the same vocabulary. Operators also need counts such as “how many
requests selected billing?”, “how many Boolean answers were true?” and “has the
urgency distribution changed?”, without deriving production metrics from
whichever Audit page happens to be open.
Deliver the complete scope below. Several implementation PRs are appropriate;
diagnostics, monitoring, metrics and the supporting capabilities remain part of
this enhancement.
h2. Shared model and ownership
*Drop route-action auditing and route-authored business metadata from the
target design.* Remove the separate route-decision concept from new capture,
configuration, the shared presentation model and the TUI/CLI/MCP workflows. Do
not retain it as an optional feature. This removes action declarations, action
vocabularies, application operation/target, policy/rule metadata, policy
reasons, action-specific filters/counters and decision-to-evidence panels.
Audit should expose what the expert returned and how the evaluation ran.
Ordinary route behavior stays the responsibility of the application and is not
represented as a semantic audit decision.
Keep *Result*, *Execution status*, *Assessment operation* and *Execution
reason* distinct. Preserve the expert's returned Boolean, choice, score or
classification, its captured meaning, expert instance/provider, definition
name, invocation origin and recorded route/exchange/breadcrumb context. A
Boolean true, classification label or high score never implies allow/block or
severity. Auditing requires no decision bean, route annotation,
business-metadata block or cross-reference beyond its ordinary configuration.
Original result interpretation remains captured automatically; it is separate
from the business annotations being dropped.
*Camel exposes information; experts and applications own its meaning.* Capture
and expose supplied interpretation, model/revision and optional task/version
identifiers as reported. Camel does not detect meaning changes, derive semantic
identities, require authors to version every question, decide whether two
populations mean the same thing, or automatically split series/reset baselines.
Meaning, versioning, grouping and comparison policies belong to the expert,
application or external monitoring system. Missing metadata is displayed as not
provided.
*The TUI is a visualization surface.* Its time windows, filters, comparisons
and display pause affect queries and presentation only. Collection, retention,
thresholds, active detector baselines and notification behavior are configured
outside the TUI. Existing audit configuration remains DSL-owned and frozen
while the service runs.
*Stores store information; calculation belongs to the producer of the
measurement.* Stores persist and retrieve audit records and already-computed
snapshots, and own retention, durability and recovery of stored information.
They do not calculate result statistics, case summaries, coverage or drift.
Camel can provide ordinary counting/formatting utilities and assemble explicit
audit links, but expert/application/monitoring producers own semantic grouping
and comparison policies. Camel exposes supplied measurements and their
availability. Memory and durable stores may offer different retention/recovery
guarantees.
Bulk retrieval has a separate responsibility boundary.
{{SemanticAuditReader.getAll(eventIds)}} retrieves records by ID without a
universal evidence-specific 100-ID cap. Console/API callers enforce
request-size and response budgets. Backend implementations manage their own
batch limits and may split retrieval into smaller storage queries. Preserve
legacy v1 evidence-record validation, existing query page limits and input
bounds. Support existing readers that still enforce the old cap through bounded
caller-side chunking during migration. Do not silently truncate requested IDs
or treat a lookup failure as a missing record. This requires runtime/SPI and
caller changes, with no new persisted field or breaking storage-schema change.
Keep responsibilities small and explicit:
||Part||Responsibility||
| Expert/adapter | Assess the submitted input and declare result meaning. |
| Route | Invoke the expert and continue normal application routing; no
separate audit decision declaration. |
| Audit runtime | Capture evaluation results, execution outcomes and available
runtime context automatically; associate evaluations, batch members and request
outcomes using their recorded identifiers. |
| Metric/monitoring producer | Own measurement semantics, grouping and
comparisons; supply measurements for exposure. The producer can be an expert,
application or external monitoring system. |
| Store | Persist and retrieve supplied records/state; implement retention and
recovery. |
| Console/JMX | Expose the shared records and computed snapshots through their
respective read interfaces. |
| TUI/CLI/MCP presentation | Format and present that data, with read-only
navigation, filters and visualization windows. |
These are responsibility boundaries, not a request for a new framework or
module for each row. Reuse existing Camel infrastructure, give each calculation
one owner, and avoid separate accumulators or monitoring logic in stores,
management interfaces and clients.
h2. Proposed scope
h3. 1. Consistent Audit, CLI and playground presentation
Introduce a shared result formatter for Boolean, choice, score and
classification, used by Audit, CLI and the playground with the appropriate
captured/submitted interpretation. Show exact fractional scores, empty label
sets, omitted results, keyed probabilities and confidence without rounding them
into policy judgments. Probability and confidence retain their own meanings.
Use bordered panels: context and visualization controls at the top, the
evaluation table in the centre, a selected-record inspector below it, and
persistent health and keyboard hints. Keep expert identity and typed result
visible. At 144x40, show Timestamp, Executing route, Evaluation,
Expert/provider, Result, Execution, Duration and Breadcrumb. The selected
evaluation's inspector presents returned result and captured meaning beside
execution/provenance details. At 100x30, retain Timestamp, Executing route,
Expert, Result and Execution in the table; move the definition name, provider
and identifiers into the single-column details, with an expanded inspector
available. At 88x24, use a compact list and a full-width scrollable inspector.
Request lifecycle records remain accessible through their invocation
relationship. Do not add route-action panels, selectors or alternate views. Use
text labels alongside colour and local timestamps with clear date/time-zone
information. The mockups illustrate this layout; they are not required to
understand the feature.
*Executing route is automatic runtime provenance.* Capture the route invoking
the evaluation from the Exchange's active route context when evaluation starts,
before expert work is dispatched. Camel already exposes this through
{{ExchangeHelper.getAtRouteId(exchange)}}. The currently stored {{routeId}},
obtained from {{exchange.getFromRouteId()}}, identifies the entry route;
preserve its meaning and existing filter behavior. Propose a separate bounded
optional {{executingRouteId}} field, subject to the compatibility rules below.
For an exchange entering {{http-entry}} and invoking an evaluation inside
{{prompt-screen}}, show Executing route {{prompt-screen}} and retain Entry
route {{http-entry}} in details. Do not reconstruct the executing route from an
exchange/breadcrumb identifier, today's route definitions or completion-time
context. No author-supplied metadata or new DSL option is required. Older
records show executing route as “not recorded”; calls without an Exchange show
“n/a.” If an Exchange has no active route context, leave that identity
unavailable rather than substituting the entry route. Missing executing-route
capture never prevents browsing existing history.
*Store both identifiers; keep the table concise.* {{exchangeId}} and
{{breadcrumbId}} already belong to the version-1 record. When both are
available, including a valid breadcrumb, retain both as separate fields in the
same audit record. Exchange ID identifies the individual Exchange; Breadcrumb
ID associates related exchanges. Auditing copies Camel's existing breadcrumb
and does not generate a replacement or substitute Exchange ID under that name.
Storage durability and retention remain backend responsibilities; viewing
history reads the stored identifiers without needing the live Exchange. This is
independent of raw-input capture and needs no new schema fields for either
identifier.
*Breadcrumb ID is the only correlation identifier shown as a table column.
Exchange ID is detail-only.* Every Audit/diagnostics detail view shows both
recorded identifiers with explicit labels and access to their full values; long
table values use visible ellipsis. At narrow widths, breadcrumb also moves into
details to keep Executing route and Result readable. An absent breadcrumb is
labelled “not recorded,” without claiming why it is missing. Missing
identifiers do not prevent audit inspection. Neither ID is a route location,
and a shared breadcrumb alone does not prove causal linkage or case
completeness.
Add visualization time presets, quoted exact filter values, selectors and
explicitly scoped refinements such as text, execution state and result
availability. Preserve the reader's exact-match contract. The existing
{{routeId}} filter remains an Entry route filter; filtering by Executing route
must use an explicitly supported query field or a clearly labelled loaded-page
refinement. Page-local results and counters must be labelled as such. Complete
window counts come from runtime-computed snapshots or a bounded runtime
calculation over sufficiently complete retained data; the store is not required
to calculate them. Stabilize live selection, report new arrivals and make
pause, older pages and expired cursors clear. Update CLI text, JSON, help and
documentation consistently.
h3. 2. Cases, relationships, health and export
Group related evaluations and request outcomes by a source-scoped breadcrumb,
with clearly labelled exchange, invocation and event fallbacks when necessary.
Show batch siblings and request-to-completion associations using captured
identifiers. Group membership is association, not proof of causation or
completeness.
Distinguish related records not yet loaded, unavailable records, omitted
results and incomplete retrieval scope. An evaluation is independently
inspectable; do not introduce missing-action or missing-business-metadata
warnings. Do not infer the reason an individual record is unavailable from a
global eviction counter.
Separate capture, reader, delivery and observed expert health. Show
per-instance collection state and execution outcomes, retained capacity,
eviction/drop counts and sink/observer failures. Keep errors visible at narrow
widths and distinguish disabled capture, no matches and a failed backend.
Support visible failure notifications and runtime-configured operational
notification hooks without giving the TUI control over them.
Provide bounded case export and structured MCP case summaries with retrieval
scope, relationship completeness and missing references. Summaries exclude
captured input. Explicit input-bearing details/exports identify untrusted data
and capture/redaction/omission state; apply the same boundary to generic
TUI-to-AI inspection paths.
h3. 3. Preserve historical meaning and support useful diagnostics
Whenever an evaluation is being audited, automatically capture its bounded,
contract-declared original interpretation when the configured store/sinks
support it: relevant choice keys/descriptions, score levels, Boolean labels and
explicit no-match or probability-domain relationships where available. No
separate interpretation-capture opt-in is required. Capture effective meaning
at invocation time; editing or deleting a definition must not reinterpret its
history. Raw-input capture retains its existing separate opt-in; this change
does not enable arbitrary prompt, instruction or parameter capture. Agree exact
bounds, field handling and omission representation before emission. Older
records remain readable, with missing interpretation labelled “not recorded”;
never reconstruct it from today's definition.
Enable new persisted interpretation/finding fields and executing-route
provenance only when the configured store/sinks declare support. Unsupported
capture is explicitly visible through health/detail responses, without sending
unrecognized fields to older implementations. Lack of extension support does
not by itself reject the existing configuration or prevent startup.
Derive diagnostics from existing results: reported top-two probability gap,
alternatives and ties, explicitly designated no-match, and score modes where a
declared mapping supports them. Preserve the selected answer even when it
differs from the highest reported probability. Mark incomplete distributions
and unavailable interpretation; diagnostics are neither model reasoning nor
calibrated certainty. No extra inference is required.
For example, a choice result selecting billing with reported probabilities
billing=0.68, technical=0.29 and other=0.03 can display the reported top-two
gap of 0.39 and technical as the next alternative. An ordinary key named other
is not automatically a no-match answer. A score of 1.6 remains 1.6; captured
level descriptions may explain its position between levels, but cannot turn it
into a discrete score of 2 or a severity.
Allow explicit transfer of permitted captured input into a new playground
draft. Identify missing original parameters and changed execution context;
never claim exact replay. Browsing and draft creation do not invoke an expert.
Evaluation remains the playground's separate, explicit Run action.
h3. 4. Typed-result metrics and expert-owned MBeans
Provide shared utilities for bounded typed-result counters/distributions and
expose expert-owned measurements independently of audit retention and TUI
polling. Measurement producers supply their dimensions and any task/version
identifiers; Camel does not derive semantic identity or manage series versions:
* *Choice:* count the selected answer once per successful evaluation; include
proportions and denominators. Probability entries are not additional answers.
* *Boolean:* count the returned true/false answer. For TypeSafe yes/no
questions, use the adapter's final Boolean; do not apply another threshold. A
reported yes-probability histogram is a separate metric. Failures and
unavailable results are not false answers.
* *Score:* count, sum, minimum/maximum and bounded numeric histograms. Preserve
fractions; exact level counts require discrete-level semantics.
* *Classification:* per-label membership and empty-set counts. Multiple labels
may contribute to one evaluation; percentages need not sum to 100%.
Count completed assessment members, including batch members, rather than
provider calls. Separate provider attempts and semantic completions,
exclusions, unavailable results and collection loss. The metric owner defines
grouping and decides how question, criterion, model or policy changes affect
its series. Camel preserves the supplied dimensions and exposes their meaning;
it does not infer semantic continuity or automatically create a new series.
Expose shared health and expert-described metric snapshots through *JMX, TUI,
CLI and MCP*, using the same read-only source. JMX remains optional at
deployment. Experts own the measurements and supply their name, meaning, kind,
unit, values and bounded dimensions. They may own a facade and contribute
provider-specific metrics without duplicating collection. Use a small optional
description/snapshot contract through the console for generic presentation; do
not introspect arbitrary MBeans or add provider-specific TUI branches. Experts
without custom metrics need not implement this extension.
Use bounded tables/buckets, lifecycle registration and no MBean per
answer/event. Console and JMX consume the same snapshot; reads do not run
inference, scan all records or synchronously query a remote store. Include
interval/epoch, units, denominators and availability. Keep runtime cumulative
counters distinct from stored historical aggregates. Honor JMX-disabled
deployments; expose no reset, threshold, baseline or lifecycle mutations on the
new metrics facade.
h3. 5. Semantic Monitoring and AiEvent integration
Add a sibling Semantic Monitoring view for measurements, comparisons and
signals supplied by an expert/application/monitoring producer, with persisted
snapshots where configured. That producer owns comparison semantics; the store
saves/retrieves supplied state and Camel exposes it. Support visualization
against a supplied fixed reference by default, and a previous-period comparison
when provided or supported by that producer. Show reference/current bounds,
sample counts, exclusions/loss, reported revisions and source-provided
comparison state. Missing information remains not provided. Camel does not
infer comparability or manage semantic changes. A distribution change does not
establish accuracy loss or a security incident and never changes routing
behavior.
Operators may change visualization periods and inspect available comparisons.
Active detector policy remains owned by its producer outside the TUI; a
visualization query does not emit operational signals. Historical windows
depend on the information produced and retained. This enhancement exposes and
visualizes monitoring information without adding a Camel semantic-change
management engine.
Use the merged common AiEvent API from CAMEL-25408 as the shared occurrence
contract for Audit and Monitoring. Add component-owned typed Semantic events
for evaluation starts/completions and request outcomes, carrying bounded
captured results, meanings, execution state and provenance. The root supports
exchange-free events; no synthetic Exchange or new core hierarchy is needed.
Keep expert identity and adapter provider in the typed payload and use a stable
publishing-family label. AiEvent is the specialized Camel event
contract/envelope that producers implement and consumers recognize. Assessment
fields deliberately belong to the Semantic specialization; the common root
remains generic. EventHelper publishes notifications and EventNotifier defines
the consumer contract. AiEvent itself does not define a wire transport, queue
or persistence format.
Construct the safe occurrence data once. Feed the typed event directly to the
existing audit recorder, which applies capture policy, attaches permitted input
privately and sends the record through its bounded sink queue. Publish the
input-free event through {{EventHelper.notifyAiEvent}} for expert/monitoring
consumers. This makes the common event contract useful to both features while
preserving audit capture independently of management-listener filters. Audit
records remain the persisted history format; do not store raw event objects or
broadcast captured input.
Result-metric producers consume typed completion events once per assessment
member. They may publish already-computed metric, comparison and signal
snapshots as typed AiEvents; Camel exposes those supplied values through the
existing read-only snapshot surfaces. Distinguish starts, completions, request
outcomes and producer outputs so counters cannot double-count or consume their
own output as another evaluation. Producers retain ownership of calculations,
grouping, comparability and notification policy. Stores persist/retrieve
supplied information, and TUI/CLI/MCP display it.
Use the merged helper's notifier lifecycle, filtering and failure isolation,
with short callbacks and bounded consumer processing. Expose measured
loss/freshness; event dispatch is synchronous, best effort and supplies no
durable replay. Monitoring consumers and event production work with audit
retention disabled and no TUI attached. Preserve existing SemanticObserver
compatibility and its parent-context/detached tracing hooks; AiEvent does not
replace those guarantees. This integration requires runtime/component work, not
a breaking persisted-schema change or an OpenTelemetry dependency in
camel-semantic. Existing tool/loop events may support their own activity
metrics, but do not fabricate semantic results or route actions from them.
h3. 6. Expert-reported findings
Support bounded expert-authored findings with their supplied meaning and
severity provenance. Missing severity means not reported; never derive it from
a Boolean, score or confidence. Findings describe the expert's assessment and
remain supplementary to its returned typed result.
Drop route-authored policy priority, policy/rule coverage and
enforcement-evidence views that depended on the removed route-decision
metadata. Business-policy experts continue to report ordinary Boolean, choice,
score or classification results and expert-owned findings through the same
generic presentation.
h2. Compatibility and implementation areas
Prefer *(a) presentation changes* and *(b) optional, versioned console
enrichment* for relationships, aggregates and capabilities. Agree *(c) bounded
additive record fields* only where historical interpretation, findings or
executing-route provenance need persistence. Both identifiers already exist:
showing Breadcrumb ID in the table and both IDs in details is *(a)*. Showing
the existing entry route is also *(a)*; capturing the separate executing route
is runtime work with a proposed *(c)* field, requiring maintainer agreement.
Keep {{routeId}} and its query semantics unchanged. Existing v1 allowlists
require capability-gated emission: keep older records readable and existing
configurations usable, and visibly report unsupported capture. The
declaration/negotiation mechanism still requires API design. *No (d) breaking
record-schema change is proposed.* Runtime collectors, store capabilities and
management projections are additional work, not presentation-only changes.
*Migration of the dropped feature:* deprecate {{SemanticAuditDecision}} and
phase out its route-decision recording/configuration path, with documentation
and examples using automatic evaluation capture. This API/configuration
lifecycle change requires maintainer agreement; any eventual API removal
follows the normal compatibility process. Preserve the ability to deserialize
and export already-persisted v1 decision records as legacy data. Do not delete
stored history, reinterpret it as an evaluation, or remove accepted fields from
the v1 storage schema. Legacy data compatibility does not restore route actions
as an optional workflow in the proposed model or UI. The runtime transition
must be documented rather than silently turning existing recording calls into
no-ops.
Build on SemanticAuditRecord, SemanticAuditQuery, SemanticAuditConsole,
SemanticAuditService, SemanticObserver, SemanticAuditView, SemanticResultView,
the semantic CLI and TUI MCP tools. Add optional capabilities with explicit
unsupported/partial states; do not impose full-store scans on the one-second
refresh or require every custom reader/expert to implement every extension.
Collection and notification failures must not change inference or routing
behavior.
The interpretation/finding/provenance compatibility behavior and
storage-versus-calculation boundary are specified above. Implementation review
must settle bounded field definitions and capability negotiation,
retrieval/persistence contracts, shared snapshot responses and supplied
metric/identity metadata. There is no Camel-owned semantic identity or
meaning-change detection contract. The store contract contains no requirement
to calculate semantic aggregates or comparisons. Exact SPI signatures, field
names and management naming follow implementation review.
h2. Acceptance criteria
# Boolean, choice, fractional score, empty classification, omitted result,
failed execution and business-policy assessment fixtures retain distinct
meanings across TUI, CLI and MCP. Shared result formatting agrees with the
playground.
# Audited evaluations automatically capture bounded original interpretation on
compatible backends, without a separate opt-in or enabling raw-input capture.
Definition edits/deletion do not change captured historical labels. Older
records remain readable with missing interpretation labelled “not recorded.”
New fields are emitted only with declared store/sink support; unsupported
capture is visible and does not prevent startup. Capture bounds and
mixed-version behavior are documented and tested.
# Route-bound and direct assessments, unavailable related records, absent
breadcrumbs, partial batches and cancellation-followed-by-completion are
navigable without fabricated results or completeness claims. Evaluation
auditing works without a decision bean, route annotation or business-metadata
configuration. Cases associate evaluations and lifecycle records through their
captured identifiers.
# The framed layouts work at 144x40, 100x30 and 88x24. Selection, input-loading
states, mixed dates, quoted filters, narrow-width errors and keyboard/help
parity are verified.
# Summary/list/metrics surfaces remain input-free. Explicit input detail, case
export and draft handoff preserve the trust boundary and never trigger
automatic inference.
# Known fixture populations produce correct choice, Boolean, numeric and
classification measurements, with batch members counted once and omissions/loss
visible. Supplied dimensions/revisions survive capture and exposure unchanged.
Camel neither infers semantic changes nor automatically splits, merges or
resets series; missing versions do not prevent ordinary audit/metric exposure.
# An expert-defined metric is visible through JMX, TUI, CLI and MCP with
matching supplied name, meaning, unit and values. Metrics are read-only,
cardinality is bounded, lifecycle cleanup works, and disabled JMX leaves
console-based functionality available. No surface independently redefines the
metric or duplicates its collection.
# Monitoring displays supplied fixed-reference and previous-period comparisons,
sample counts and source-reported unavailable/incomparable states. Stores
persist/retrieve supplied information; console/JMX expose the same snapshots.
Memory/durable recovery behavior and absent metadata are explicit. Camel does
not infer changes in meaning, impose a versioning scheme or manage comparison
policy. TUI visualization changes do not alter collection, detector state or
notifications.
# Typed Semantic AiEvents feed audit recording and expert/monitoring consumers
for route-bound and direct evaluations, with audit-disabled monitoring and no
TUI required. Each completed batch member is counted once; starts, request
outcomes and supplied monitoring outputs are distinct. Audit capture/input
remain private and independent of management-listener filters; history remains
readable without an event consumer. Verify listener failure/queue-loss
behavior, supplied snapshot exposure and compatibility with existing observer
tracing hooks.
# Expert-reported findings retain their author, meaning and supplied severity
scale; absent severity is explicit. Documentation and examples cover a fixed
classifier, an instruction-driven expert and provider-independent
classification/business-policy data without route-authored policy metadata or
coverage claims.
# A nested-route fixture entering route A and invoking an evaluation in route B
records Entry route A and Executing route B at invocation time, without
additional author configuration. Completion, route renaming/deletion and
historical browsing do not change those recorded identities. Existing
{{routeId}} filters keep their entry-route meaning. Direct calls, missing
active route context, old records and unsupported capture are represented
without guessing a location or preventing normal audit operation.
# A record with both valid identifiers retains {{exchangeId}} and
{{breadcrumbId}} together through storage and retrieval. Breadcrumb ID is the
only correlation column in the wide table; Exchange ID is never a table column.
Detail boxes expose both full recorded values. At narrow widths both IDs remain
accessible in details while Executing route and Result stay readable. Absent
breadcrumbs are handled without generating or relabelling identifiers.
Summary/list responses remain input-free.
# Route-action auditing is dropped from the target capture/configuration model
and TUI/CLI/MCP presentation, including its metadata, filters, counters, panels
and policy-coverage dependencies. Migration deprecates the existing recording
entry point and replaces its examples with automatic evaluation auditing.
Already-persisted v1 decision records remain deserializable/exportable as
legacy data without restoring the removed feature or breaking the storage
schema.
# Bulk retrieval accepts a permitted request exceeding 100 IDs without an
evidence-specific SPI limit. Caller request/response budgets are enforced,
backend chunking returns all available requested records, and existing readers
with the old cap remain usable during migration. Missing records remain
distinct from retrieval failures. Legacy v1 evidence-record validation, query
page limits and input bounds remain unchanged.
h2. Related work and visual references
* [CAMEL-25499: semantic audit
foundation|https://issues.apache.org/jira/browse/CAMEL-25499] — follow-up to
the delivered implementation.
* [CAMEL-25357: semantic dev
console|https://issues.apache.org/jira/browse/CAMEL-25357] — overlapping
foundational proposal; coordinate scope and reuse the existing console/recorder.
* [CAMEL-25408: common AiEvent
API|https://issues.apache.org/jira/browse/CAMEL-25408] and [PR
#27714|https://github.com/apache/camel/pull/27714] — Resolved/Fixed and merged,
verified 2026-10-13; the common API is available on main for 4.24.0. Build the
Audit/Monitoring event integration on this foundation; no pending upstream
hierarchy/dispatch request remains.
* [CAMEL-23369: GenAI
observability|https://issues.apache.org/jira/browse/CAMEL-23369] — coordinate
common telemetry integration; this issue adds semantic result/case/monitoring
meaning.
* [Original framed layout
reference|https://issues.apache.org/jira/secure/attachment/13085107/13085107_camel-semantic-audit-tui.png]
— layout inspiration only; the proposed mockups use the information model
described here.
The attached mockups illustrate Audit, diagnostics and Monitoring at 144x40 and
100x30. They contain synthetic examples, not implemented UI screenshots. At
88x24, use a compact list and full-width scrollable inspector.
_AI-assisted content prepared by Codex on behalf of Luigi De Masi
(@luigidemasi)._
> camel-semantic - enhance audit UX, result metrics and drift monitoring
> ----------------------------------------------------------------------
>
> Key: CAMEL-25618
> URL: https://issues.apache.org/jira/browse/CAMEL-25618
> Project: Camel
> Issue Type: Improvement
> Components: camel-ai, camel-jbang, camel-management
> Reporter: Luigi De Masi
> Assignee: Luigi De Masi
> Priority: Major
> Attachments: semantic-audit-framed-100x30.png,
> semantic-audit-framed-144x40.png, semantic-diagnostics-framed-100x30.png,
> semantic-diagnostics-framed-144x40.png,
> semantic-monitoring-framed-100x30.png, semantic-monitoring-framed-144x40.png
>
>
> h2. Motivation
> Build on the semantic auditing delivered by CAMEL-25499 to give operators a
> consistent way to inspect the expert's returned result and the route where
> the evaluation was invoked, follow related evaluations, and observe changes
> in results over time. Simplify semantic audit to automatically captured
> evaluations, their execution lifecycle and runtime provenance.
> A fixed-purpose injection classifier, an instruction-driven choice or score
> expert, and a moderation or business-policy expert should be understandable
> through the same vocabulary. Operators also need counts such as “how many
> requests selected billing?”, “how many Boolean answers were true?” and “has
> the urgency distribution changed?”, without deriving production metrics from
> whichever Audit page happens to be open.
> Deliver the complete scope below in [PR
> #27791|https://github.com/apache/camel/pull/27791], organized into four
> reviewable commits: explicit expert requirements, audit foundation and
> presentation, shared monitoring and presentation, and playground/layout
> controls. Diagnostics, monitoring, metrics and the supporting capabilities
> remain part of this enhancement. Intended fix version: *4.24.0*.
> h2. Shared model and ownership
> *Drop route-action auditing and route-authored business metadata from the
> target design.* Remove the separate route-decision concept from new capture,
> configuration, the shared presentation model and the TUI/CLI/MCP workflows.
> Do not retain it as an optional feature. This removes action declarations,
> action vocabularies, application operation/target, policy/rule metadata,
> policy reasons, action-specific filters/counters and decision-to-evidence
> panels. Audit should expose what the expert returned and how the evaluation
> ran. Ordinary route behavior stays the responsibility of the application and
> is not represented as a semantic audit decision.
> Keep *Result*, *Execution status*, *Assessment operation* and *Execution
> reason* distinct. Preserve the expert's returned Boolean, choice, score or
> classification, its captured meaning, expert instance/provider, definition
> name, invocation origin and recorded route/exchange/breadcrumb context. A
> Boolean true, classification label or high score never implies allow/block or
> severity. Auditing requires no decision bean, route annotation,
> business-metadata block or cross-reference beyond its ordinary configuration.
> Original result interpretation remains captured automatically; it is separate
> from the business annotations being dropped.
> *Camel exposes information; experts and applications own its meaning.*
> Capture and expose supplied interpretation, model/revision and optional
> task/version identifiers as reported. Camel does not detect meaning changes,
> derive semantic identities, require authors to version every question, decide
> whether two populations mean the same thing, or automatically split
> series/reset baselines. Meaning, versioning, grouping and comparison policies
> belong to the expert, application or external monitoring system. Missing
> metadata is displayed as not provided.
> *The TUI is a visualization surface.* Its time windows, filters, comparisons
> and display pause affect queries and presentation only. Collection,
> retention, thresholds, active detector baselines and notification behavior
> are configured outside the TUI. Existing audit configuration remains
> DSL-owned and frozen while the service runs.
> *Stores store information; calculation belongs to the producer of the
> measurement.* Stores persist and retrieve audit records and already-computed
> snapshots, and own retention, durability and recovery of stored information.
> They do not calculate result statistics, case summaries, coverage or drift.
> Camel can provide ordinary counting/formatting utilities and assemble
> explicit audit links, but expert/application/monitoring producers own
> semantic grouping and comparison policies. Camel exposes supplied
> measurements and their availability. Memory and durable stores may offer
> different retention/recovery guarantees.
> Bulk retrieval has a separate responsibility boundary.
> {{SemanticAuditReader.getAll(eventIds)}} retrieves records by ID without a
> universal evidence-specific 100-ID cap. Console/API callers enforce
> request-size and response budgets. Backend implementations manage their own
> batch limits and may split retrieval into smaller storage queries. Preserve
> legacy v1 evidence-record validation, existing query page limits and input
> bounds. Support existing readers that still enforce the old cap through
> bounded caller-side chunking during migration. Do not silently truncate
> requested IDs or treat a lookup failure as a missing record. This requires
> runtime/SPI and caller changes, with no new persisted field or breaking
> storage-schema change.
> Keep responsibilities small and explicit:
> ||Part||Responsibility||
> | Expert/adapter | Assess the submitted input and declare result meaning. |
> | Route | Invoke the expert and continue normal application routing; no
> separate audit decision declaration. |
> | Audit runtime | Capture evaluation results, execution outcomes and
> available runtime context automatically; associate evaluations, batch members
> and request outcomes using their recorded identifiers. |
> | Metric/monitoring producer | Own measurement semantics, grouping and
> comparisons; supply measurements for exposure. The producer can be an expert,
> application or external monitoring system. |
> | Store | Persist and retrieve supplied records/state; implement retention
> and recovery. |
> | Console/JMX | Expose the shared records and computed snapshots through
> their respective read interfaces. |
> | TUI/CLI/MCP presentation | Format and present that data, with read-only
> navigation, filters and visualization windows. |
> These are responsibility boundaries, not a request for a new framework or
> module for each row. Reuse existing Camel infrastructure, give each
> calculation one owner, and avoid separate accumulators or monitoring logic in
> stores, management interfaces and clients.
> h2. Proposed scope
> h3. 1. Consistent Audit, CLI and playground presentation
> Introduce a shared result formatter for Boolean, choice, score and
> classification, used by Audit, CLI and the playground with the appropriate
> captured/submitted interpretation. Show exact fractional scores, empty label
> sets, omitted results, keyed probabilities and confidence without rounding
> them into policy judgments. Probability and confidence retain their own
> meanings.
> Use bordered panels: context and visualization controls at the top, the
> evaluation table in the centre, a selected-record inspector below it, and
> persistent health and keyboard hints. Keep expert identity and typed result
> visible. At 144x40, show Timestamp, Executing route, Evaluation,
> Expert/provider, Result, Execution, Duration and Breadcrumb. The selected
> evaluation's inspector presents returned result and captured meaning beside
> execution/provenance details. At 100x30, retain Timestamp, Executing route,
> Expert, Result and Execution in the table; move the definition name, provider
> and identifiers into the single-column details, with an expanded inspector
> available. At 88x24, use a compact list and a full-width scrollable
> inspector. Request lifecycle records remain accessible through their
> invocation relationship. Do not add route-action panels, selectors or
> alternate views. Use text labels alongside colour and local timestamps with
> clear date/time-zone information. The mockups illustrate this layout; they
> are not required to understand the feature.
> *Executing route is automatic runtime provenance.* Capture the route invoking
> the evaluation from the Exchange's active route context when evaluation
> starts, before expert work is dispatched. Camel already exposes this through
> {{ExchangeHelper.getAtRouteId(exchange)}}. The currently stored {{routeId}},
> obtained from {{exchange.getFromRouteId()}}, identifies the entry route;
> preserve its meaning and existing filter behavior. Propose a separate bounded
> optional {{executingRouteId}} field, subject to the compatibility rules
> below. For an exchange entering {{http-entry}} and invoking an evaluation
> inside {{prompt-screen}}, show Executing route {{prompt-screen}} and retain
> Entry route {{http-entry}} in details. Do not reconstruct the executing route
> from an exchange/breadcrumb identifier, today's route definitions or
> completion-time context. No author-supplied metadata or new DSL option is
> required. Older records show executing route as “not recorded”; calls without
> an Exchange show “n/a.” If an Exchange has no active route context, leave
> that identity unavailable rather than substituting the entry route. Missing
> executing-route capture never prevents browsing existing history.
> *Store both identifiers; keep the table concise.* {{exchangeId}} and
> {{breadcrumbId}} already belong to the version-1 record. When both are
> available, including a valid breadcrumb, retain both as separate fields in
> the same audit record. Exchange ID identifies the individual Exchange;
> Breadcrumb ID associates related exchanges. Auditing copies Camel's existing
> breadcrumb and does not generate a replacement or substitute Exchange ID
> under that name. Storage durability and retention remain backend
> responsibilities; viewing history reads the stored identifiers without
> needing the live Exchange. This is independent of raw-input capture and needs
> no new schema fields for either identifier.
> *Breadcrumb ID is the only correlation identifier shown as a table column.
> Exchange ID is detail-only.* Every Audit/diagnostics detail view shows both
> recorded identifiers with explicit labels and access to their full values;
> long table values use visible ellipsis. At narrow widths, breadcrumb also
> moves into details to keep Executing route and Result readable. An absent
> breadcrumb is labelled “not recorded,” without claiming why it is missing.
> Missing identifiers do not prevent audit inspection. Neither ID is a route
> location, and a shared breadcrumb alone does not prove causal linkage or case
> completeness.
> Add visualization time presets, quoted exact filter values, selectors and
> explicitly scoped refinements such as text, execution state and result
> availability. Preserve the reader's exact-match contract. The existing
> {{routeId}} filter remains an Entry route filter; filtering by Executing
> route must use an explicitly supported query field or a clearly labelled
> loaded-page refinement. Page-local results and counters must be labelled as
> such. Complete window counts come from runtime-computed snapshots or a
> bounded runtime calculation over sufficiently complete retained data; the
> store is not required to calculate them. Stabilize live selection, report new
> arrivals and make pause, older pages and expired cursors clear. Update CLI
> text, JSON, help and documentation consistently.
> h3. 2. Cases, relationships, health and export
> Group related evaluations and request outcomes by a source-scoped breadcrumb,
> with clearly labelled exchange, invocation and event fallbacks when
> necessary. Show batch siblings and request-to-completion associations using
> captured identifiers. Group membership is association, not proof of causation
> or completeness.
> Distinguish related records not yet loaded, unavailable records, omitted
> results and incomplete retrieval scope. An evaluation is independently
> inspectable; do not introduce missing-action or missing-business-metadata
> warnings. Do not infer the reason an individual record is unavailable from a
> global eviction counter.
> Separate capture, reader, delivery and observed expert health. Show
> per-instance collection state and execution outcomes, retained capacity,
> eviction/drop counts and sink/observer failures. Keep errors visible at
> narrow widths and distinguish disabled capture, no matches and a failed
> backend. Support visible failure notifications and runtime-configured
> operational notification hooks without giving the TUI control over them.
> Provide bounded case export and structured MCP case summaries with retrieval
> scope, relationship completeness and missing references. Summaries exclude
> captured input. Explicit input-bearing details/exports identify untrusted
> data and capture/redaction/omission state; apply the same boundary to generic
> TUI-to-AI inspection paths.
> h3. 3. Preserve historical meaning and support useful diagnostics
> Capture the complete contract-declared original interpretation once in the
> typed semantic event: relevant choice keys/descriptions, score levels,
> Boolean labels and explicit no-match or probability-domain relationships
> where available. Freeze effective meaning at invocation time; editing or
> deleting a definition must not reinterpret that event or its retained
> history. There is no producer-side interpretation truncation limit. Each
> retaining consumer enforces its own payload, queue and storage budget. The
> audit recorder creates a bounded projection before enqueueing,
> capability-gates additive fields for its store/sinks, and reports explicit
> omission reasons when its budget or capabilities prevent retaining
> interpretation. No separate interpretation-capture opt-in is required.
> Raw-input capture retains its existing separate opt-in; interpretation
> capture does not enable arbitrary prompt, instruction or parameter capture.
> Older records remain readable, with missing interpretation labelled “not
> recorded”; never reconstruct it from today's definition.
> Enable new persisted interpretation/finding fields and executing-route
> provenance only when the configured store/sinks declare support. Unsupported
> capture is explicitly visible through health/detail responses, without
> sending unrecognized fields to older implementations. Lack of extension
> support does not by itself reject the existing configuration or prevent
> startup.
> Derive diagnostics from existing results: reported top-two probability gap,
> alternatives and ties, explicitly designated no-match, and score modes where
> a declared mapping supports them. Preserve the selected answer even when it
> differs from the highest reported probability. Mark incomplete distributions
> and unavailable interpretation; diagnostics are neither model reasoning nor
> calibrated certainty. No extra inference is required.
> For example, a choice result selecting billing with reported probabilities
> billing=0.68, technical=0.29 and other=0.03 can display the reported top-two
> gap of 0.39 and technical as the next alternative. An ordinary key named
> other is not automatically a no-match answer. A score of 1.6 remains 1.6;
> captured level descriptions may explain its position between levels, but
> cannot turn it into a discrete score of 2 or a severity.
> Allow explicit transfer of permitted captured input into a new playground
> draft. Identify missing original parameters and changed execution context;
> never claim exact replay. Browsing and draft creation do not invoke an
> expert. Evaluation remains the playground's separate, explicit Run action.
> h3. 4. Typed-result metrics and expert-owned MBeans
> Provide shared utilities for bounded typed-result counters/distributions and
> expose expert-owned measurements independently of audit retention and TUI
> polling. Measurement producers supply their dimensions and any task/version
> identifiers; Camel does not derive semantic identity or manage series
> versions:
> * *Choice:* count the selected answer once per successful evaluation; include
> proportions and denominators. Probability entries are not additional answers.
> * *Boolean:* count the returned true/false answer. For TypeSafe yes/no
> questions, use the adapter's final Boolean; do not apply another threshold. A
> reported yes-probability histogram is a separate metric. Failures and
> unavailable results are not false answers.
> * *Score:* count, sum, minimum/maximum and bounded numeric histograms.
> Preserve fractions; exact level counts require discrete-level semantics.
> * *Classification:* per-label membership and empty-set counts. Multiple
> labels may contribute to one evaluation; percentages need not sum to 100%.
> Count completed assessment members, including batch members, rather than
> provider calls. Separate provider attempts and semantic completions,
> exclusions, unavailable results and collection loss. The metric owner defines
> grouping and decides how question, criterion, model or policy changes affect
> its series. Camel preserves the supplied dimensions and exposes their
> meaning; it does not infer semantic continuity or automatically create a new
> series.
> Expose shared health and expert-described metric snapshots through *JMX, TUI,
> CLI and MCP*, using the same read-only source. JMX remains optional at
> deployment. Experts own the measurements and supply their name, meaning,
> kind, unit, values and bounded dimensions. They may own a facade and
> contribute provider-specific metrics without duplicating collection. Use a
> small optional description/snapshot contract through the console for generic
> presentation; do not introspect arbitrary MBeans or add provider-specific TUI
> branches. Experts without custom metrics need not implement this extension.
> Use bounded tables/buckets, lifecycle registration and no MBean per
> answer/event. Console and JMX consume the same snapshot; reads do not run
> inference, scan all records or synchronously query a remote store. Include
> interval/epoch, units, denominators and availability. Keep runtime cumulative
> counters distinct from stored historical aggregates. Honor JMX-disabled
> deployments; expose no reset, threshold, baseline or lifecycle mutations on
> the new metrics facade.
> h3. 5. Semantic Monitoring and AiEvent integration
> Add a sibling Semantic Monitoring view for measurements, comparisons and
> signals supplied by an expert/application/monitoring producer, with persisted
> snapshots where configured. That producer owns comparison semantics; the
> store saves/retrieves supplied state and Camel exposes it. Support
> visualization against a supplied fixed reference by default, and a
> previous-period comparison when provided or supported by that producer. Show
> reference/current bounds, sample counts, exclusions/loss, reported revisions
> and source-provided comparison state. Missing information remains not
> provided. Camel does not infer comparability or manage semantic changes. A
> distribution change does not establish accuracy loss or a security incident
> and never changes routing behavior.
> Operators may change visualization periods and inspect available comparisons.
> Active detector policy remains owned by its producer outside the TUI; a
> visualization query does not emit operational signals. Historical windows
> depend on the information produced and retained. This enhancement exposes and
> visualizes monitoring information without adding a Camel semantic-change
> management engine. Latest snapshots refresh once per second. Sampled history
> and persistence retain changed snapshots at most once per minute per series
> by default, with an immediate first sample. A registered
> SemanticMonitoringConfiguration bean exposes a runtime-changeable
> historyInterval in seconds (1..86400). Explicit producer-published
> snapshots/comparisons/signals bypass that sampling interval and are retained
> immediately when changed. Unchanged samples do not create history or store
> writes. Expose actual retained coverage and distinguish local capture times
> from producer-reported windows; recovered history with unavailable local
> capture times must say so. No durable monitoring backend is bundled.
> Use the merged common AiEvent API from CAMEL-25408 as the shared occurrence
> contract for Audit and Monitoring. Add component-owned typed Semantic events
> for evaluation starts/completions and request outcomes, carrying captured
> results, complete frozen interpretation, execution state and provenance.
> Route-bound evaluation notifications implement the existing
> {{AiEvent.ExchangeAiEvent}} contract and expose the live Exchange to standard
> exchange-aware listeners. Detached direct calls and request outcomes use
> exchange-free events; never create a synthetic Exchange. No new core event
> hierarchy is needed. Keep expert identity and adapter provider in the typed
> payload and use a stable publishing-family label. AiEvent is the specialized
> Camel event contract/envelope that producers implement and consumers
> recognize. Assessment fields deliberately belong to the Semantic
> specialization; the common root remains generic. EventHelper publishes
> notifications and EventNotifier defines the consumer contract. AiEvent itself
> does not define a wire transport, queue or persistence format.
> Construct and freeze the occurrence data once. Feed the typed event directly
> to the existing audit recorder, which applies capture policy and its own
> retention budget, attaches permitted captured input privately and sends the
> frozen record through its bounded sink queue. Publish that same event through
> {{EventHelper.notifyAiEvent}} for expert/monitoring consumers. The typed
> data, JSON representation and safe string summary exclude raw input, but a
> route-bound event deliberately exposes its live Exchange (including body and
> headers) through {{getExchange()}}/{{getSource()}}. This is an in-process
> listener contract, not a promise that the whole event is input-free.
> Listeners must keep callbacks short, avoid payload dumps, and freeze/project
> any data they retain; audit stores, queues and monitoring snapshots never
> retain the live Exchange or raw event. Audit capture remains independent of
> management-listener filters. Audit records remain the persisted history
> format.
> Result-metric producers consume typed completion events once per assessment
> member. They may publish already-computed metric, comparison and signal
> snapshots as typed AiEvents; Camel exposes those supplied values through the
> existing read-only snapshot surfaces. Distinguish starts, completions,
> request outcomes and producer outputs so counters cannot double-count or
> consume their own output as another evaluation. Producers retain ownership of
> calculations, grouping, comparability and notification policy. Stores
> persist/retrieve supplied information, and TUI/CLI/MCP display it.
> Use the merged helper's notifier lifecycle, filtering and failure isolation,
> with short callbacks and bounded consumer processing. Expose measured
> loss/freshness; event dispatch is synchronous, best effort and supplies no
> durable replay. Monitoring consumers and event production work with audit
> retention disabled and no TUI attached. Preserve existing SemanticObserver
> compatibility and its parent-context/detached tracing hooks; AiEvent does not
> replace those guarantees. This integration requires runtime/component work,
> not a breaking persisted-schema change or an OpenTelemetry dependency in
> camel-semantic. Existing tool/loop events may support their own activity
> metrics, but do not fabricate semantic results or route actions from them.
> h3. 6. Expert-reported findings
> Support bounded expert-authored findings with their supplied meaning and
> severity provenance. Missing severity means not reported; never derive it
> from a Boolean, score or confidence. Findings describe the expert's
> assessment and remain supplementary to its returned typed result.
> Drop route-authored policy priority, policy/rule coverage and
> enforcement-evidence views that depended on the removed route-decision
> metadata. Business-policy experts continue to report ordinary Boolean,
> choice, score or classification results and expert-owned findings through the
> same generic presentation.
> h2. Compatibility and implementation areas
> Prefer *(a) presentation changes* and *(b) optional, versioned console
> enrichment* for relationships, aggregates and capabilities. Agree *(c)
> bounded additive record fields* only where historical interpretation,
> findings or executing-route provenance need persistence. Both identifiers
> already exist: showing Breadcrumb ID in the table and both IDs in details is
> *(a)*. Showing the existing entry route is also *(a)*; capturing the separate
> executing route is runtime work with a proposed *(c)* field, requiring
> maintainer agreement. Keep {{routeId}} and its query semantics unchanged.
> Existing v1 allowlists require capability-gated emission: keep older records
> readable and existing store/sink configurations usable, and visibly report
> unsupported capture. The declaration/negotiation mechanism still requires API
> design. *No (d) breaking record-schema change is proposed.* Runtime
> collectors, store capabilities and management projections are additional
> work, not presentation-only changes.
> *Migration of the dropped preview feature:* remove {{SemanticAuditDecision}}
> and its route-decision recording/configuration path outright, with
> documentation and examples using automatic evaluation capture. Semantic and
> expert APIs are preview APIs and may change without a deprecation period;
> document these removals in the 4.24 upgrade guide. Preserve the ability to
> deserialize and export already-persisted v1 decision records as legacy data.
> Do not delete stored history, reinterpret it as an evaluation, or remove
> accepted fields from the v1 storage schema. Legacy compatibility does not
> restore route actions as an optional workflow. Do not silently turn old
> recording calls into no-ops.
> *Explicit experts:* require an explicit registry expert on each evaluation.
> Remove language default-expert configuration, sole-expert fallback,
> block-level expert inheritance and adapter service discovery. Providers
> remain available through explicitly registered SemanticExpert/SemanticAdapter
> beans. Update declarations, schemas, resolver/tooling metadata, documentation
> and the upgrade guide consistently. These are agreed preview
> API/configuration changes, not removals of the expert/adapter extension
> contracts themselves.
> Build on SemanticAuditRecord, SemanticAuditQuery, SemanticAuditConsole,
> SemanticAuditService, SemanticObserver, SemanticAuditView,
> SemanticResultView, the semantic CLI and TUI MCP tools. Add optional
> capabilities with explicit unsupported/partial states; do not impose
> full-store scans on the one-second refresh or require every custom
> reader/expert to implement every extension. Collection and notification
> failures must not change inference or routing behavior.
> The interpretation/finding/provenance compatibility behavior and
> storage-versus-calculation boundary are specified above. Implementation
> review must settle bounded field definitions and capability negotiation,
> retrieval/persistence contracts, shared snapshot responses and supplied
> metric/identity metadata. There is no Camel-owned semantic identity or
> meaning-change detection contract. The store contract contains no requirement
> to calculate semantic aggregates or comparisons. Exact SPI signatures, field
> names and management naming follow implementation review.
> h2. Acceptance criteria
> # Boolean, choice, fractional score, empty classification, omitted result,
> failed execution and business-policy assessment fixtures retain distinct
> meanings across TUI, CLI and MCP. Shared result formatting agrees with the
> playground.
> # Typed events carry complete frozen interpretation without producer-side
> truncation. Audited evaluations automatically retain bounded projections on
> compatible backends, without a separate opt-in or enabling raw-input capture;
> each retaining consumer owns and enforces its budgets. Definition
> edits/deletion do not change captured historical labels. Older records remain
> readable with missing interpretation labelled “not recorded.” New fields are
> emitted only with declared store/sink support; unsupported capture is visible
> and does not prevent startup. Capture bounds and mixed-version behavior are
> documented and tested.
> # Route-bound and direct assessments, unavailable related records, absent
> breadcrumbs, partial batches and cancellation-followed-by-completion are
> navigable without fabricated results or completeness claims. Evaluation
> auditing works without a decision bean, route annotation or business-metadata
> configuration. Cases associate evaluations and lifecycle records through
> their captured identifiers.
> # The framed layouts work at 144x40, 100x30 and 88x24. Selection,
> input-loading states, mixed dates, quoted filters, narrow-width errors and
> keyboard/help parity are verified.
> # Summary/list/metrics surfaces remain input-free. Explicit input detail,
> case export and draft handoff preserve the trust boundary and never trigger
> automatic inference.
> # Known fixture populations produce correct choice, Boolean, numeric and
> classification measurements, with batch members counted once and
> omissions/loss visible. Supplied dimensions/revisions survive capture and
> exposure unchanged. Camel neither infers semantic changes nor automatically
> splits, merges or resets series; missing versions do not prevent ordinary
> audit/metric exposure.
> # An expert-defined metric is visible through JMX, TUI, CLI and MCP with
> matching supplied name, meaning, unit and values. Metrics are read-only,
> cardinality is bounded, lifecycle cleanup works, and disabled JMX leaves
> console-based functionality available. No surface independently redefines the
> metric or duplicates its collection.
> # Monitoring displays supplied fixed-reference and previous-period
> comparisons, sample counts and source-reported unavailable/incomparable
> states. Stores persist/retrieve supplied information; console/JMX expose the
> same snapshots. Memory/durable recovery behavior and absent metadata are
> explicit. Camel does not infer changes in meaning, impose a versioning scheme
> or manage comparison policy. TUI visualization changes do not alter
> collection, detector state or notifications.
> # Typed Semantic AiEvents feed audit recording and expert/monitoring
> consumers for route-bound and direct evaluations, with audit-disabled
> monitoring and no TUI required. Each completed batch member is counted once;
> starts, request outcomes and supplied monitoring outputs are distinct.
> Route-bound notifications expose the live Exchange through ExchangeAiEvent;
> detached notifications do not fabricate one. The frozen data/JSON and summary
> surfaces exclude raw input, while opt-in audit input capture remains
> independent of management-listener filters; history remains readable without
> an event consumer. Verify listener failure/queue-loss behavior, supplied
> snapshot exposure and compatibility with existing observer tracing hooks.
> # Expert-reported findings retain their author, meaning and supplied severity
> scale; absent severity is explicit. Documentation and examples cover a fixed
> classifier, an instruction-driven expert and provider-independent
> classification/business-policy data without route-authored policy metadata or
> coverage claims.
> # A nested-route fixture entering route A and invoking an evaluation in route
> B records Entry route A and Executing route B at invocation time, without
> additional author configuration. Completion, route renaming/deletion and
> historical browsing do not change those recorded identities. Existing
> {{routeId}} filters keep their entry-route meaning. Direct calls, missing
> active route context, old records and unsupported capture are represented
> without guessing a location or preventing normal audit operation.
> # A record with both valid identifiers retains {{exchangeId}} and
> {{breadcrumbId}} together through storage and retrieval. Breadcrumb ID is the
> only correlation column in the wide table; Exchange ID is never a table
> column. Detail boxes expose both full recorded values. At narrow widths both
> IDs remain accessible in details while Executing route and Result stay
> readable. Absent breadcrumbs are handled without generating or relabelling
> identifiers. Summary/list responses remain input-free.
> # Route-action auditing is dropped from the target capture/configuration
> model and TUI/CLI/MCP presentation, including its metadata, filters,
> counters, panels and policy-coverage dependencies. Migration removes the
> preview recording entry point outright and replaces its examples with
> automatic evaluation auditing; the 4.24 upgrade guide documents the removal.
> Already-persisted v1 decision records remain deserializable/exportable as
> legacy data without restoring the removed feature or breaking the storage
> schema.
> # Bulk retrieval accepts a permitted request exceeding 100 IDs without an
> evidence-specific SPI limit. Caller request/response budgets are enforced,
> backend chunking returns all available requested records, and existing
> readers with the old cap remain usable during migration. Missing records
> remain distinct from retrieval failures. Legacy v1 evidence-record
> validation, query page limits and input bounds remain unchanged.
> # Every evaluation names its expert registry bean explicitly. Remove default
> experts, sole-expert fallback, block-level experts and adapter service
> discovery; migrate Java/XML/YAML declarations, validator schemas, tooling,
> providers, examples and upgrade guidance. Missing expert references fail
> clearly. This requirement does not remove the SemanticExpert or
> SemanticAdapter extension contracts.
> # Monitoring latest data refreshes every second, while changed sampled
> history/store writes default to one minute per series with runtime
> configuration. Explicit publications are retained immediately when changed.
> Tests cover notifier filtering, audit-disabled event eligibility, restart
> resets, retention cadence, runtime cadence changes and actual retained
> coverage.
> h2. Related work and visual references
> * [CAMEL-25499: semantic audit
> foundation|https://issues.apache.org/jira/browse/CAMEL-25499] — follow-up to
> the delivered implementation.
> * [CAMEL-25357: semantic dev
> console|https://issues.apache.org/jira/browse/CAMEL-25357] — overlapping
> foundational proposal; coordinate scope and reuse the existing
> console/recorder.
> * [CAMEL-25408: common AiEvent
> API|https://issues.apache.org/jira/browse/CAMEL-25408] and [PR
> #27714|https://github.com/apache/camel/pull/27714] — Resolved/Fixed and
> merged, verified 2026-10-13; the common API is available on main for 4.24.0.
> Build the Audit/Monitoring event integration on this foundation; no pending
> upstream hierarchy/dispatch request remains.
> * [CAMEL-23369: GenAI
> observability|https://issues.apache.org/jira/browse/CAMEL-23369] — coordinate
> common telemetry integration; this issue adds semantic result/case/monitoring
> meaning.
> * [Original framed layout
> reference|https://issues.apache.org/jira/secure/attachment/13085107/13085107_camel-semantic-audit-tui.png]
> — layout inspiration only; the proposed mockups use the information model
> described here.
> The attached mockups illustrate Audit, diagnostics and Monitoring at 144x40
> and 100x30. They contain synthetic examples, not implemented UI screenshots.
> At 88x24, use a compact list and full-width scrollable inspector.
> _AI-assisted content prepared by Codex on behalf of Luigi De Masi
> (@luigidemasi)._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
