[
https://issues.apache.org/jira/browse/FLINK-35984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leonard Xu updated FLINK-35984:
-------------------------------
Affects Version/s: cdc-3.1.1
> Job crashes when metadata column names present in transform rules
> -----------------------------------------------------------------
>
> Key: FLINK-35984
> URL: https://issues.apache.org/jira/browse/FLINK-35984
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Affects Versions: cdc-3.1.1
> Reporter: yux
> Assignee: MOBIN
> Priority: Major
> Labels: pull-request-available
> Fix For: cdc-3.2.0
>
>
> Given such a transform rule:
> transform:
> projection: \*, '__namespace_name__schema_name__table_name__' AS
> string_literal
> Obviously this shouldn't insert any metadata columns. However since metadata
> column existence check was done by searching identifier string in statement,
> without considering any syntax info:
> ```java
> // TransformParser.java#L357
> if (transformStatement.contains(DEFAULT_NAMESPACE_NAME) ...
> ```
> Transform operator will mistakenly append metadata columns into Janino
> arguments list, and crash the job:
> Caused by: java.lang.IllegalArgumentException: wrong number of arguments
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.codehaus.janino.ExpressionEvaluator.evaluate(ExpressionEvaluator.java:541)
> at
> org.codehaus.janino.ExpressionEvaluator.evaluate(ExpressionEvaluator.java:533)
> at
> org.apache.flink.cdc.runtime.operators.transform.ProjectionColumnProcessor.evaluate(ProjectionColumnProcessor.java:64)
> at
> org.apache.flink.cdc.runtime.operators.transform.TransformProjectionProcessor.processData(TransformProjectionProcessor.java:153)
> at
> org.apache.flink.cdc.runtime.operators.transform.TransformDataOperator.processProjection(TransformDataOperator.java:387)
> at
> org.apache.flink.cdc.runtime.operators.transform.TransformDataOperator.processDataChangeEvent(TransformDataOperator.java:328)
> at
> org.apache.flink.cdc.runtime.operators.transform.TransformDataOperator.processElement(TransformDataOperator.java:190)
> at
> org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:237)
> at
> org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:146)
> at
> org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:110)
> at
> org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807)
> at
> org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953)
> at
> org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
> at java.lang.Thread.run(Thread.java:748)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
