[
https://issues.apache.org/jira/browse/CAMEL-24172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18097916#comment-18097916
]
Omar Atie edited comment on CAMEL-24172 at 7/21/26 4:51 PM:
------------------------------------------------------------
CAMEL-24172 — Two-Phase Plan
Split the ticket into *{*}two PRs{*}{*}: producer first, consumer second.
**DataFormat{*}* (`apicurio-avro`) stays out of scope for both — it’s a
separate feature/JIRA.
—
Phase 1 — Producer MVP (Registry management from routes)
*{*}Goal:{*}* Let routes call Apicurio Registry v3 via the official Java SDK
without custom beans or raw HTTP.
|Area|Scope|
|------|--------|
|*{*}New module{*}*|`components/camel-apicurio-registry/`|
|*{*}URI{*}*|`apicurio-registry:groupId/artifactId?registryUrl=...`|
|*{*}Operations (producer){*}*|`createArtifact`, `updateArtifact`,
`deleteArtifact`, `getArtifactContent`, `getArtifactMetadata`,
`searchArtifacts`, `testCompatibility`|
|*{*}Config{*}*|`registryUrl`, auth (token/OAuth as SDK supports),
`artifactType`, `ifExists`, timeouts|
|*{*}Headers{*}*|`CamelApicurioRegistryOperation`, group/artifact/version/type,
compatibility result, etc.|
|*{*}SDK integration{*}*|Wrapper around `apicurio-registry-java-sdk`
(Vert.x/Kiota lifecycle in endpoint start/stop)|
|*{*}Tests{*}*|Mock SDK unit tests + Testcontainers IT against Apicurio
Registry|
|*{*}Docs{*}*|Component page, operations page, catalog/generated metadata,
upgrade guide note|
*{*}Covers JIRA use cases:{*}* 1, 2, 4, 6 (schema registration, fetch spec,
migration, CI compatibility gate)
*{*}Out of scope for Phase 1:{*}* consumer, webhooks, DataFormat
*{*}Reference components:{*}* `camel-couchdb` (structure), `camel-consul`
(operations/headers pattern)
—
Phase 2 — Consumer (Event-driven schema awareness)
*{*}Goal:{*}* React to registry changes from routes — polling and/or webhooks.
|Area|Scope|
|------|--------|
|*{*}Polling consumer{*}*|`delay`-based poll for new artifact versions; emit
exchange with version/metadata headers|
|*{*}Webhook consumer{*}* (if feasible)|HTTP endpoint to receive registry
lifecycle events (may lean on `platform-http` or embedded handler)|
|*{*}Producer polish{*}*|Any gaps from Phase 1 review (extra ops, error
mapping, auth edge cases)|
|*{*}Tests{*}*|Consumer unit tests + IT (poll detects new version; webhook
receives event)|
|*{*}Docs{*}*|Consumer section, examples for schema-change notifications|
*{*}Covers JIRA use cases:{*}* 3, 5 (schema evolution notifications,
AsyncAPI-driven refresh)
*{*}Out of scope for Phase 2:{*}* DataFormat marshalling/unmarshalling
—
Summary table
| |Phase 1|Phase 2|
|—|---------|---------|
|*{*}Focus{*}*|Producer — CRUD + search + compatibility|Consumer — poll +
webhook|
|*{*}PR size{*}*|Medium–large (new component)|Medium|
|*{*}User value{*}*|Manage schemas from Camel routes immediately|Notify/react
when schemas change|
|*{*}Use cases{*}*|1, 2, 4, 6|3, 5|
|*{*}Deferred{*}*|Consumer, webhooks|DataFormat (future ticket)|
—
One-line summary
- *{*}Phase 1:{*}* Build `camel-apicurio-registry` as a *{*}producer-only{*}*
component for artifact CRUD, search, and compatibility testing.
- *{*}Phase 2:{*}* Add *{*}consumer{*}* support (poll + webhook) so routes can
react to schema/version changes.
—
Suggested JIRA comment (optional)
> Proposing a two-phase delivery for CAMEL-24172:
> *{*}Phase 1{*}* — producer MVP
> (create/update/delete/get/search/testCompatibility + tests/docs).
> *{*}Phase 2{*}* — consumer (polling for new versions + webhook
> notifications).
> DataFormat (`apicurio-avro`) deferred to a follow-up ticket.
> Phase 1 PR to follow after assignment.
was (Author: JIRAUSER313980):
## CAMEL-24172 — Two-Phase Plan
Split the ticket into **two PRs**: producer first, consumer second.
**DataFormat** (`apicurio-avro`) stays out of scope for both — it’s a separate
feature/JIRA.
---
### Phase 1 — Producer MVP (Registry management from routes)
**Goal:** Let routes call Apicurio Registry v3 via the official Java SDK
without custom beans or raw HTTP.
| Area | Scope |
|------|--------|
| **New module** | `components/camel-apicurio-registry/` |
| **URI** | `apicurio-registry:groupId/artifactId?registryUrl=...` |
| **Operations (producer)** | `createArtifact`, `updateArtifact`,
`deleteArtifact`, `getArtifactContent`, `getArtifactMetadata`,
`searchArtifacts`, `testCompatibility` |
| **Config** | `registryUrl`, auth (token/OAuth as SDK supports),
`artifactType`, `ifExists`, timeouts |
| **Headers** | `CamelApicurioRegistryOperation`, group/artifact/version/type,
compatibility result, etc. |
| **SDK integration** | Wrapper around `apicurio-registry-java-sdk`
(Vert.x/Kiota lifecycle in endpoint start/stop) |
| **Tests** | Mock SDK unit tests + Testcontainers IT against Apicurio Registry
|
| **Docs** | Component page, operations page, catalog/generated metadata,
upgrade guide note |
**Covers JIRA use cases:** 1, 2, 4, 6 (schema registration, fetch spec,
migration, CI compatibility gate)
**Out of scope for Phase 1:** consumer, webhooks, DataFormat
**Reference components:** `camel-couchdb` (structure), `camel-consul`
(operations/headers pattern)
---
### Phase 2 — Consumer (Event-driven schema awareness)
**Goal:** React to registry changes from routes — polling and/or webhooks.
| Area | Scope |
|------|--------|
| **Polling consumer** | `delay`-based poll for new artifact versions; emit
exchange with version/metadata headers |
| **Webhook consumer** (if feasible) | HTTP endpoint to receive registry
lifecycle events (may lean on `platform-http` or embedded handler) |
| **Producer polish** | Any gaps from Phase 1 review (extra ops, error mapping,
auth edge cases) |
| **Tests** | Consumer unit tests + IT (poll detects new version; webhook
receives event) |
| **Docs** | Consumer section, examples for schema-change notifications |
**Covers JIRA use cases:** 3, 5 (schema evolution notifications,
AsyncAPI-driven refresh)
**Out of scope for Phase 2:** DataFormat marshalling/unmarshalling
---
### Summary table
| | Phase 1 | Phase 2 |
|---|---------|---------|
| **Focus** | Producer — CRUD + search + compatibility | Consumer — poll +
webhook |
| **PR size** | Medium–large (new component) | Medium |
| **User value** | Manage schemas from Camel routes immediately | Notify/react
when schemas change |
| **Use cases** | 1, 2, 4, 6 | 3, 5 |
| **Deferred** | Consumer, webhooks | DataFormat (future ticket) |
---
### One-line summary
- **Phase 1:** Build `camel-apicurio-registry` as a **producer-only** component
for artifact CRUD, search, and compatibility testing.
- **Phase 2:** Add **consumer** support (poll + webhook) so routes can react to
schema/version changes.
---
### Suggested JIRA comment (optional)
> Proposing a two-phase delivery for CAMEL-24172:
> **Phase 1** — producer MVP (create/update/delete/get/search/testCompatibility
> + tests/docs).
> **Phase 2** — consumer (polling for new versions + webhook notifications).
> DataFormat (`apicurio-avro`) deferred to a follow-up ticket.
> Phase 1 PR to follow after assignment.
> New component proposal: camel-apicurio-registry
> -----------------------------------------------
>
> Key: CAMEL-24172
> URL: https://issues.apache.org/jira/browse/CAMEL-24172
> Project: Camel
> Issue Type: New Feature
> Reporter: Carles Arnal
> Priority: Major
>
> I'd like to propose a new component for integrating with *Apicurio Registry.*
> Camel has Confluent Schema Registry support, but nothing for Apicurio, which
> is widely used in the Kafka ecosystem. Users currently have to write custom
> beans or raw HTTP calls to interact with it from Camel routes.
> The idea is a camel-apicurio-registry component built on the official Java
> SDK (available in Maven Central) that would support:
> - *Producer ops:* create/update/delete artifacts, get content/metadata,
> search, test compatibility
> - *Consumer:* poll for new versions or consume webhook notifications on
> artifact lifecycle events
>
> apicurio-registry:my-group/my-schema?registryUrl=[http://localhost:8080/apis/registry/v3]
> I'm opening this based on the conversation in [#camel > New component
> proposal: camel-apicurio-registry @
> 💬|https://camel.zulipchat.com/#narrow/channel/257298-camel/topic/New.20component.20proposal.3A.20camel-apicurio-registry/near/611041432].
> h2. *Use Cases*
> {*}Use Case 1: Schema Governance in Kafka Pipelines{*}{*}{{*}}
> Register and validate schemas before producing Kafka messages. Ensures all
> producers use a centrally governed schema, preventing schema drift across
> teams.
> {code:java}
> // Register an Avro schema before starting to produce
> from("file:schemas?include=.*\\.avsc")
> .setHeader("CamelApicurioRegistryOperation", constant("createArtifact"))
> .setHeader("CamelApicurioRegistryGroupId", constant("kafka-schemas"))
> .setHeader("CamelApicurioRegistryArtifactType", constant("AVRO"))
> .setHeader("CamelApicurioRegistryIfExists", constant("CREATE_VERSION"))
> .to("apicurio-registry://kafka-schemas?registryUrl={{registry.url}}")
> .log("Registered schema: ${header.CamelApicurioRegistryArtifactId}");
> {code}
> {*}Use Case 2: Contract-First API Development{*}{*}{{*}}
> Pull an OpenAPI spec from the registry and use it to validate incoming HTTP
> requests, ensuring API implementations stay in sync with the contract.
> {code:java}
> // Fetch the latest OpenAPI spec from the registry and validate requests
> from("timer:refreshSpec?period=60000")
> .setHeader("CamelApicurioRegistryOperation",
> constant("getArtifactContent"))
>
> .to("apicurio-registry://my-apis/orders-api?registryUrl={{registry.url}}")
> .convertBodyTo(String.class)
> .to("direct:updateValidator");
> from("platform-http:/api/orders")
> .to("direct:validateAgainstSpec")
> .to("bean:orderService");
> {code}
> {*}Use Case 3: Schema Evolution Notifications{*}{*}{{*}}
> Poll the registry for new artifact versions and notify downstream systems
> (Slack, email, CI/CD pipelines) when schemas change — essential for data mesh
> architectures.
> {code:java}
> // React to new schema versions
>
> from("apicurio-registry://my-group/payment-event?registryUrl={{registry.url}}&delay=30000")
> .log("New version of payment-event schema:
> v${header.CamelApicurioRegistryVersion}")
> .choice()
> .when(header("CamelApicurioRegistryArtifactType").isEqualTo("AVRO"))
> .to("slack:#schema-changes")
> .when(header("CamelApicurioRegistryArtifactType").isEqualTo("OPENAPI"))
> .to("webhook:https://ci.example.com/rebuild-sdk";)
> .end();
> {code}
> {*}Use Case 4: Multi-Format Schema Migration{*}{*}{{*}}
> Migrate schemas across environments (dev → staging → prod) or convert between
> formats. Useful for organizations managing schemas across multiple clusters.
> {code:java}
> // Export all schemas from one registry and import into another
>
> from("apicurio-registry://my-group?registryUrl={{registry.dev.url}}&operation=searchArtifacts")
> .split(body())
> .setHeader("CamelApicurioRegistryOperation",
> constant("getArtifactContent"))
>
> .toD("apicurio-registry://${header.CamelApicurioRegistryGroupId}/${header.CamelApicurioRegistryArtifactId}?registryUrl={{registry.dev.url}}")
> .setHeader("CamelApicurioRegistryOperation",
> constant("createArtifact"))
> .setHeader("CamelApicurioRegistryIfExists",
> constant("CREATE_VERSION"))
>
> .to("apicurio-registry://my-group?registryUrl={{registry.prod.url}}")
> .end()
> .log("Schema migration complete");
> {code}
>
> {*}Use Case 5: AsyncAPI-Driven Event Router{*}{*}{{*}}
> Use AsyncAPI specs stored in the registry to dynamically configure Camel
> routes for event-driven architectures — the registry becomes the source of
> truth for messaging topology.
> {code:java}
> // Read an AsyncAPI spec and route events to the correct channel
> from("timer:configRefresh?period=120000")
> .setHeader("CamelApicurioRegistryOperation",
> constant("getArtifactContent"))
>
> .to("apicurio-registry://event-platform/order-events?registryUrl={{registry.url}}&artifactType=ASYNCAPI")
> .bean("asyncApiRouteBuilder", "updateRoutes");
>
> {code}
> {*}Use Case 6: Schema Compatibility Gate in CI/CD{*}{*}{{*}}
> Integrate into a Tekton/Jenkins pipeline to test schema compatibility before
> merging changes. Fail the build if a new schema version would break consumers.
> {code:java}
> // CI/CD compatibility check
> from("direct:checkCompatibility")
> .setHeader("CamelApicurioRegistryOperation",
> constant("testCompatibility"))
>
> .to("apicurio-registry://my-group/user-event?registryUrl={{registry.url}}")
> .choice()
> .when(header("CamelApicurioRegistryCompatible").isEqualTo(true))
> .log("Schema is compatible — safe to deploy")
> .to("direct:proceed")
> .otherwise()
> .log("BREAKING CHANGE detected!")
> .to("direct:failBuild")
> .end();
>
> {code}
> {*}Use Case 7: Data Format — Schema-Validated Marshalling{*}{*}{{*}}
> Use Apicurio Registry as a Camel DataFormat for serialization/deserialization
> in any route (not just Kafka), pulling schemas from the registry at runtime.
>
> {code:java}
> // Unmarshal JMS messages using Avro schema from the registry
> from("jms:queue:orders")
> .unmarshal().custom("apicurio-avro:my-group/order-schema")
> .process(exchange -> {
> GenericRecord order = exchange.getIn().getBody(GenericRecord.class);
> // process the order...
> })
> .marshal().custom("apicurio-avro:my-group/order-response-schema")
> .to("jms:queue:order-responses");
> {code}
> {*}Supported Artifact Types{*}{*}{{*}}
> Apicurio Registry supports a wide range of artifact types beyond traditional
> schemas, making the component useful across many integration scenarios.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
