[
https://issues.apache.org/jira/browse/FLINK-39171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-39171:
-----------------------------------
Labels: pull-request-available (was: )
> Oracle pipeline transform fails with quoted uppercase projection columns
> ------------------------------------------------------------------------
>
> Key: FLINK-39171
> URL: https://issues.apache.org/jira/browse/FLINK-39171
> Project: Flink
> Issue Type: Improvement
> Components: Flink CDC
> Affects Versions: cdc-3.6.0
> Environment: * Flink: 1.20.1
> * Oracle: 19c (Non-CDB,)
> * Startup mode: {{initial}}
> * Flink CDC：3.6.0
> Reporter: tianfy
> Priority: Major
> Labels: pull-request-available
>
> In Oracle CDC {*}pipeline mode{*}, transform projection can fail when
> projection uses quoted uppercase source columns (for example {{{}`ID`{}}}),
> with:
> {{Column 'ID' not found in any table}}
> Observed behavior suggests Oracle pipeline schema columns are normalized to
> lowercase internally (for example {{{}id{}}}), while transform SQL validation
> treats quoted identifiers as exact/case-sensitive, causing mismatch.
> h3. Reproduction
> # Create Oracle table (unquoted column names, Oracle stores as uppercase):
> {{ TESTUSER.ORDERS(ID, PRODUCT_ID, QUANTITY, UPDATED_AT)}}
> 2. Run Oracle pipeline with transform projection using quoted
> uppercase source columns, e.g.:
> pipeline:
> schema.change.behavior: LENIENT
>
> transform:
> - source-table: ORCL19.TESTUSER.ORDERS
> primary-keys: id
> projection: "`ID` as `id`,`PRODUCT_ID` as `product_id`,`QUANTITY` as
> `quantity`,`UPDATED_AT` as `updated_at`"
>
> source:
> type: oracle
> database: ORCL19
> tables: ORCL19.TESTUSER.ORDERS
> scan.startup.mode: initial
>
> 3. Start the job.
> h3. Actual behavior
> Job fails during post-transform schema handling:
> *
> org.apache.flink.cdc.runtime.operators.transform.exceptions.TransformException
> * Caused by: org.apache.calcite.sql.validate.SqlValidatorException: Column
> 'ID' not found in any table
> Stack path includes: PostTransformOperator ->
> TransformParser.generateProjectionColumns -> Calcite validator
> h3. Expected behavior
> Oracle pipeline should use a consistent column-name strategy between schema
> and transform projection:
> # Either preserve Oracle column case consistently, or
> 2. Normalize both schema and projection consistently, or
> 3. Resolve quoted identifiers safely for Oracle semantics.
> At minimum, projection generated from source metadata should not fail with
> column-not-found because of internal case normalization mismatch.
> h3. Impact
> * Oracle pipeline job fails at startup for common projection patterns using
> original DB column names.
> * Blocks end-to-end sync in transform-enabled pipelines.
> h3. Workaround
> * Convert Oracle source field names in projection to lowercase before
> submission (for example {{id}} instead of {{{}ID{}}}).
> * Avoid quoted uppercase source identifiers in transform projection.
> h3. Additional notes
> If needed, I can provide a minimal runnable reproducer (DDL + full pipeline
> YAML + complete stack trace).
>
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
