[
https://issues.apache.org/jira/browse/CAMEL-24977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-24977 started by Luigi De Masi.
---------------------------------------------
> Add provider-agnostic semantic evaluation across Camel EIPs
> -----------------------------------------------------------
>
> Key: CAMEL-24977
> URL: https://issues.apache.org/jira/browse/CAMEL-24977
> Project: Camel
> Issue Type: New Feature
> Components: camel-ai, camel-core, camel-yaml-dsl, eip
> Reporter: Luigi De Masi
> Assignee: Luigi De Masi
> Priority: Major
>
> h2. Motivation
> Provide a provider-agnostic semantic evaluation API and Camel language so
> route authors can ask named questions about message content and use the
> resulting boolean decisions, categories, or scores across EIPs. For example,
> a support message can be classified as billing, technical, or other; a
> separate question can assess urgency or whether the message contains an
> actionable request.
> CAMEL-24871 added the Jev integration. This proposal builds on that work by
> separating the common question/evaluation contract from provider-specific
> implementations and adding a generic selector-based Choice capability. Jev
> can implement the common SPI, and other hosted or local classifiers can
> implement it without requiring Jev wire compatibility.
> The desired authoring experience is declarative: named questions in the same
> YAML file as routes, automatic provider-adapter discovery, and direct use
> through Camel's language integration. Route authors should not need to
> instantiate a predicate or adapter bean for the normal case.
> This is a design proposal. The syntax below illustrates the intended
> behavior; public names, packaging and exact DSL/schema details require
> community review. It does not claim that the proposed semantic language or
> selector-based Choice is already available.
> h2. Common semantic evaluation contract
> * Define provider-independent question and result types for categorical
> Choice, yes/no judgments (including probability-to-boolean decision policy),
> and scoring against descriptive ordered levels. Preserve the useful shape of
> Jev Choice/Noul/Score through its adapter; final common type names should not
> force other providers to adopt Jev terminology.
> * Separate the selected category/value, probabilities, provider confidence,
> uncertainty policy and operational errors. A probability, a confidence
> measure and a guarantee of correctness are not interchangeable. Preserve
> provider metadata where useful without requiring all providers to report the
> same optional fields.
> * A provider component advertises an adapter implementing the common SPI. The
> adapter maps questions/state to its backend, validates supported
> capabilities, and maps typed results back to the common contract.
> * Expose evaluations through Camel's existing {{Language}}, {{Expression}}
> and {{Predicate}} contracts. Predicate use requires a boolean decision or
> explicit comparison/threshold policy; a category string is not implicitly a
> predicate.
> * Keep provider credentials, model selection, transport/inference
> configuration, and provider-specific resources with the implementing
> component. Define thread safety, bounded evaluation, cancellation and
> lifecycle behavior. Synchronous Predicate/Expression evaluation must document
> any blocking inference call.
> h2. Named questions and state in YAML
> Support a dedicated top-level {{semantic.question}} declaration from the
> initial version, alongside route definitions. The following is proposed
> syntax:
> {code}
> - semantic:
> question:
> department:
> type: choice
> state: ${header.myState}
> instructions: Which department should handle this message?
> criteria:
> billing: Invoices, payments, and refunds
> technical: Bugs, outages, and technical problems
> other: Everything else
> - route:
> id: ticket-routing
> from:
> uri: direct:tickets
> steps:
> - choice:
> selector:
> language:
> language: semantic
> expression: "ref:department"
> when:
> - value: billing
> steps:
> - to: direct:billing
> - value: technical
> steps:
> - to: direct:technical
> otherwise:
> steps:
> - to: direct:general
> {code}
> Each question owns its input-state selector. An optional language default can
> be configured:
> {code}
> camel.language.semantic.default-state=${body}
> {code}
> Proposed precedence: question {{state}}, then configured {{default-state}},
> then built-in {{${body}}}. Use Camel Simple expressions for these selectors
> initially. Compile/initialize selectors during route setup and evaluate
> against the current exchange at runtime. Only an absent selector inherits the
> default: a malformed selector or an explicitly selected header that evaluates
> to null must not silently cause different data to be classified. Input values
> are data, not recursively evaluated expressions.
> Question declarations need loader, validation, schema and tooling support.
> Resolve references before traffic where possible; reject duplicate names,
> unknown references and unsupported question capabilities. Define behavior
> across multiple route resources and reload without retaining obsolete
> definitions. Loading/validating declarations must not perform inference.
> h2. Adapter discovery and explicit selection
> The language must not require a provider endpoint URI such as
> {{jev:decisions}}. With exactly one advertised adapter implementation
> available, discover and select it automatically. With none, report a missing
> provider; with multiple, report the candidates and require explicit
> selection. Do not choose by classpath order or construct every provider
> during discovery.
> Allow an explicit reference to an existing registered instance:
> {code}
> camel.language.semantic.adapter=#bean:mySemanticAdapter
> {code}
> Alternatively, accept a fully qualified class name:
> {code}
> camel.language.semantic.adapter=com.example.MySemanticAdapter
> {code}
> The example class is illustrative. Support Camel's
> {{#class:com.example.MySemanticAdapter}} spelling as an equivalent. For a
> class-selected or discovered adapter, the semantic infrastructure should
> resolve and type-check the class through Camel, instantiate it using the
> context Injector, register it once for the language configuration, and
> integrate its lifecycle. Reuse referenced beans without duplicating their
> existing lifecycle ownership. Define registry ownership, collision handling
> and startup-failure cleanup; binding an object does not itself establish
> service lifecycle management.
> Provider configuration must remain effective even when the route contains no
> provider endpoint. For example, a Jev adapter must honor the configured Jev
> component's credentials and connection options. A manually declared adapter
> bean is an optional customization path, not a prerequisite for normal
> operation.
> h2. Generic selector-based Choice in Camel core
> Extend Choice to evaluate one ordinary Camel Expression and match its result
> against literal branch values. Core should know only expression evaluation
> and value matching; it must have no dependency on semantic questions, Jev, or
> a model runtime.
> Proposed Java shape, where {{department}} is a Camel Expression resolving the
> named semantic question:
> {code:java}
> from("direct:tickets")
> .choice(department)
> .when("billing")
> .to("direct:billing")
> .when("technical")
> .to("direct:technical")
> .otherwise()
> .to("direct:general")
> .end();
> {code}
> The same extension must work with ordinary expressions such as
> {{header("department")}}. Existing predicate-based Choice behavior must
> remain unchanged.
> * Evaluate the selector once per entry into the Choice block, including when
> the first branch does not match. Nested choices have independent evaluation
> scopes.
> * A later loop/retry entry may evaluate again. Do not introduce an indefinite
> exchange-wide semantic-result cache.
> * Define literal typing/equality, null/unmatched results, duplicate values,
> and whether/how selector branches can coexist with predicate branches.
> * A successful unmatched value selects {{otherwise}}. Inference or
> invalid-response failures follow Camel error handling; uncertainty requires
> an explicit policy.
> * Define interaction with Choice precondition mode. Message-dependent
> inference must not execute against a dummy startup exchange. CAMEL-17755
> concerns startup preconditions and is distinct from this per-message selector
> proposal.
> * Include model, reifier, runtime, Java DSL, YAML/XML serialization and
> generated schema/catalog coverage. A language plugin alone cannot add the
> proposed Choice API.
> h2. Evaluation across other EIPs
> The following are candidate uses, not a requirement to introduce
> semantic-specific changes to every EIP.
> || EIP || Example benefit || Integration ||
> | Validate | Does this bug report describe an actionable problem? | A boolean
> decision supplies the validation predicate; negative decisions use normal
> validation failure handling. |
> | Set Header / Set Property | Attach department, sentiment or urgency to a
> message. | Store a category or score for explicit reuse by subsequent
> processors. |
> | Aggregate: grouping | Group feedback by delivery, pricing or usability. |
> Use a category as part of the correlation key, retaining tenant/case
> boundaries and bounded aggregation windows where needed. |
> | Aggregate: completion | Does the accumulated conversation contain enough
> information to proceed? | Evaluate a boolean completion predicate against
> selected aggregate state, alongside a size or timeout completion condition. |
> | Recipient List | Send a complaint to support and the relevant product team.
> | Map a category, or multiple independent decisions, to configured
> recipients. |
> | Routing Slip | Select a document-processing sequence. | Map a document
> category to a predefined sequence of endpoints. |
> | Enrich | Choose the appropriate knowledge source for a support request. |
> Map a topic to a configured enrichment endpoint; ordinary code retrieves and
> merges the additional data. |
> | Loop | Continue refinement while the updated output remains incomplete. |
> Reevaluate a semantic predicate against current state, with an explicit
> iteration/time budget. |
> | Sort | Order a batch of tickets by assessed urgency. | Score each item
> once, then compare stored scores with a deterministic comparator. |
> Validate, metadata assignment and Aggregate can consume Predicate/Expression
> results directly. Recipient List, Routing Slip and Enrich require a mapping
> from semantic labels to route-author-defined destinations. The classifier
> does not invent endpoint addresses or processing steps.
> For aggregate completion, the aggregation strategy must expose the relevant
> accumulated content; the classifier does not implicitly remember earlier
> exchanges. The default completion predicate sees the aggregated exchange,
> while {{eagerCheckCompletion}} uses the incoming exchange. State selection
> must match that behavior.
> For Sort, the EIP expression selects the collection and the comparator
> determines ordering. Inference belongs before comparison. For loops or
> changed input, explicit result reuse must not hide relevant state changes.
> h2. Acceptance criteria and initial coverage
> # A common provider-independent question/result contract and adapter SPI
> support direct semantic Expression/Predicate use without requiring manual
> predicate instances.
> # A YAML file can declare named questions under {{semantic.question}} and
> reference them from routes. Loader, schema, validation and tooling support
> agree on the proposed configuration.
> # Adapter discovery, bean-reference override and class-name override are
> tested, including missing/ambiguous providers, wrong types, instance reuse,
> configuration and lifecycle behavior.
> # Question state overrides the language default; omitting both uses the body.
> Invalid selectors and missing selected state produce useful errors without
> silent input substitution.
> # Generic selector-based Choice evaluates once per block entry, works with
> semantic and non-semantic expressions, preserves existing predicate-based
> Choice, and has nested/looped/error-path coverage and documented precondition
> behavior.
> # Demonstrate Choice, Filter, Validate, Set Property and Aggregate
> integration. Document the other candidate EIPs and their mapping/state
> requirements without assuming every EIP needs a core change.
> # Provide a Jev adapter and validate the contract with deterministic test
> adapters, including a second implementation and capability mismatch cases. Do
> not claim interchangeability of model quality, probabilities or calibration.
> # Tests cover meaningful result mappings, invocation counts,
> threshold/uncertainty policy, explicit result reuse, changed-state
> reevaluation, timeouts/failures and preservation of message content. Routine
> tests require no live model credentials.
> # Document the supported property namespaces and runtime configuration paths.
> Coordinate with the generic-language starter work in CAMEL-24913 and
> CAMEL-24914 rather than assuming language configuration is generated
> automatically.
> h2. Boundaries and references
> Choosing a local inference runtime (pure JVM/WASM versus native inference),
> model weights, or a particular hosted service is outside this proposal's
> initial decision. A provider can implement Jev API compatibility, but the
> common SPI does not require it and does not imply equivalent proprietary
> model behavior or calibration. The SPI's final package, discovery metadata
> and public syntax remain open to design review.
> Related work:
> * CAMEL-24871 — Jev component and semantic EIP integration background.
> * CAMEL-24913 — Jev Spring Boot starter configuration.
> * CAMEL-24914 — Starter generation for languages backed by the generic model.
> * CAMEL-17755 — Existing Choice precondition design.
> * [Camel Language
> SPI|https://github.com/apache/camel/blob/main/core/camel-api/src/main/java/org/apache/camel/spi/Language.java]
> * [Choice
> model|https://github.com/apache/camel/blob/main/core/camel-core-model/src/main/java/org/apache/camel/model/ChoiceDefinition.java]
> * [Aggregate
> EIP|https://camel.apache.org/components/next/eips/aggregate-eip.html]
> * [System One Models|https://systemonemodels.org/]
> _AI-generated by Codex on behalf of
> [luigidemasi|https://github.com/luigidemasi]._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
