Xiqian Yu created FLINK-38540:
---------------------------------
Summary: Fix potential CME error in TransformExpressionCompiler
Key: FLINK-38540
URL: https://issues.apache.org/jira/browse/FLINK-38540
Project: Flink
Issue Type: Bug
Components: Flink CDC
Affects Versions: cdc-3.5.0
Reporter: Xiqian Yu
Just noticed very rare failure in UdfE2eITCase with the following exception
stack:
{code}
Caused by: org.apache.flink.util.FlinkRuntimeException: Failed to compile
expression TransformExpressionKey{expression='import static
org.apache.flink.cdc.runtime.functions.impl.ArithmeticFunctions.*;import static
org.apache.flink.cdc.runtime.functions.impl.CastingFunctions.*;import static
org.apache.flink.cdc.runtime.functions.impl.ComparisonFunctions.*;import static
org.apache.flink.cdc.runtime.functions.impl.LogicalFunctions.*;import static
org.apache.flink.cdc.runtime.functions.impl.StringFunctions.*;import static
org.apache.flink.cdc.runtime.functions.impl.TemporalFunctions.*;$0',
argumentNames=[$0, __time_zone__, __epoch_time__], argumentClasses=[class
java.lang.String, class java.lang.String, class java.lang.Long],
returnClass=class java.lang.String, columnNameMap={VERSION=$0}}
at
org.apache.flink.cdc.runtime.operators.transform.TransformExpressionCompiler.compileExpression(TransformExpressionCompiler.java:88)
~[blob_p-dcef4c05469f1ec41c8cff62eec3acdbed064dd4-acc6987827519710a75bf959e2073b32:3.5-SNAPSHOT]
at
org.apache.flink.cdc.runtime.operators.transform.ProjectionColumnProcessor.<init>(ProjectionColumnProcessor.java:63)
~[blob_p-dcef4c05469f1ec41c8cff62eec3acdbed064dd4-acc6987827519710a75bf959e2073b32:3.5-SNAPSHOT]
at
org.apache.flink.cdc.runtime.operators.transform.ProjectionColumnProcessor.of(ProjectionColumnProcessor.java:75)
~[blob_p-dcef4c05469f1ec41c8cff62eec3acdbed064dd4-acc6987827519710a75bf959e2073b32:3.5-SNAPSHOT]
at
org.apache.flink.cdc.runtime.operators.transform.TransformProjectionProcessor.lambda$createProjectionColumnProcessors$1(TransformProjectionProcessor.java:102)
~[blob_p-dcef4c05469f1ec41c8cff62eec3acdbed064dd4-acc6987827519710a75bf959e2073b32:3.5-SNAPSHOT]
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?]
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
~[?:?]
at
java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
~[?:?]
at
java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
~[?:?]
at
java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at
java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]
at
org.apache.flink.cdc.runtime.operators.transform.TransformProjectionProcessor.createProjectionColumnProcessors(TransformProjectionProcessor.java:109)
~[blob_p-dcef4c05469f1ec41c8cff62eec3acdbed064dd4-acc6987827519710a75bf959e2073b32:3.5-SNAPSHOT]
at
org.apache.flink.cdc.runtime.operators.transform.TransformProjectionProcessor.<init>(TransformProjectionProcessor.java:77)
~[blob_p-dcef4c05469f1ec41c8cff62eec3acdbed064dd4-acc6987827519710a75bf959e2073b32:3.5-SNAPSHOT]
at
org.apache.flink.cdc.runtime.operators.transform.PostTransformOperator.getProjectionProcessor(PostTransformOperator.java:521)
~[blob_p-dcef4c05469f1ec41c8cff62eec3acdbed064dd4-acc6987827519710a75bf959e2073b32:3.5-SNAPSHOT]
at
org.apache.flink.cdc.runtime.operators.transform.PostTransformOperator.processDataChangeEvent(PostTransformOperator.java:375)
~[blob_p-dcef4c05469f1ec41c8cff62eec3acdbed064dd4-acc6987827519710a75bf959e2073b32:3.5-SNAPSHOT]
at
org.apache.flink.cdc.runtime.operators.transform.PostTransformOperator.processElementInternal(PostTransformOperator.java:234)
~[blob_p-dcef4c05469f1ec41c8cff62eec3acdbed064dd4-acc6987827519710a75bf959e2073b32:3.5-SNAPSHOT]
at
org.apache.flink.cdc.runtime.operators.transform.PostTransformOperator.processElement(PostTransformOperator.java:152)
~[blob_p-dcef4c05469f1ec41c8cff62eec3acdbed064dd4-acc6987827519710a75bf959e2073b32:3.5-SNAPSHOT]
... 13 more
Caused by: java.util.ConcurrentModificationException
at java.util.ArrayList.checkForComodification(ArrayList.java:605) ~[?:?]
at java.util.ArrayList.equalsArrayList(ArrayList.java:599) ~[?:?]
at java.util.ArrayList.equals(ArrayList.java:561) ~[?:?]
at
{code}
Seems some internal `List` fields in TransformExpressionKey are accidentally
modified in TransformExpressionCompiler due to accidental object reuse.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
