[
https://issues.apache.org/jira/browse/FLINK-37348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leonard Xu resolved FLINK-37348.
--------------------------------
Resolution: Fixed
Fixed via master: 8019fd3fc07f3b3d4288347302b65b32860410f5
> [pipeline-paimon]fix paimon ArrayIndexOutOfBoundsException when add column
> before first_column
> ----------------------------------------------------------------------------------------------
>
> Key: FLINK-37348
> URL: https://issues.apache.org/jira/browse/FLINK-37348
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Affects Versions: cdc-3.3.0
> Reporter: MOBIN
> Assignee: MOBIN
> Priority: Major
> Labels: pull-request-available
> Fix For: cdc-3.4.0
>
>
> Repeat step：
> # ALTER TABLE table_name add COLUMN new_first_column bigint first
> # throw ArrayIndexOutOfBoundsException
> log:
> {code:java}
> 2025-02-19 11:32:16,586 INFO
> org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator [] -
> All sink subtask have flushed for table dw_app.cdc_sink19. Start to apply
> schema change request:
> SchemaChangeRequest{tableId=dw_app.cdc_sink19,
> schemaChangeEvent=AddColumnEvent{tableId=dw_app.cdc_sink19,
> addedColumns=[ColumnWithPosition{column=`new_first_column` BIGINT,
> position=BEFORE, existedColumnName=first_column}]}, subTaskId=0}
> that extracts to:
> AddColumnEvent{tableId=rt_ods.cdc_sink19_add_column_EVOLVE4,
> addedColumns=[ColumnWithPosition{column=`new_first_column` BIGINT,
> position=BEFORE, existedColumnName=first_column}]}
> 2025-02-19 11:32:16,716 ERROR
> org.apache.flink.cdc.runtime.operators.schema.common.SchemaRegistry [] - An
> exception was triggered from Schema change applying task. Job will fail now.
> org.apache.flink.util.FlinkRuntimeException: Failed to apply schema change
> event.
> at
> org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.lambda$startSchemaChangesEvolve$0(SchemaCoordinator.java:290)
> ~[flink-cdc-dist-3.3.0.jar:3.3.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [?:1.8.0_65]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_65]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [?:1.8.0_65]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [?:1.8.0_65]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_65]
> Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
> at java.util.ArrayList.elementData(ArrayList.java:418) ~[?:1.8.0_65]
> at java.util.ArrayList.get(ArrayList.java:431) ~[?:1.8.0_65]
> at
> org.apache.flink.cdc.connectors.paimon.sink.PaimonMetadataApplier.applyAddColumnWithBeforePosition(PaimonMetadataApplier.java:275)
> ~[flink-cdc-pipeline-connector-paimon-3.3.0.jar:3.3.0]
> at
> org.apache.flink.cdc.connectors.paimon.sink.PaimonMetadataApplier.applyAddColumnEventWithPosition(PaimonMetadataApplier.java:237)
> ~[flink-cdc-pipeline-connector-paimon-3.3.0.jar:3.3.0]
> at
> org.apache.flink.cdc.connectors.paimon.sink.PaimonMetadataApplier.applyAddColumn(PaimonMetadataApplier.java:206)
> ~[flink-cdc-pipeline-connector-paimon-3.3.0.jar:3.3.0]
> at
> org.apache.flink.cdc.connectors.paimon.sink.PaimonMetadataApplier.lambda$applySchemaChange$0(PaimonMetadataApplier.java:127)
> ~[flink-cdc-pipeline-connector-paimon-3.3.0.jar:3.3.0]
> at
> org.apache.flink.cdc.common.event.visitor.SchemaChangeEventVisitor.visit(SchemaChangeEventVisitor.java:47)
> ~[flink-cdc-dist-3.3.0.jar:3.3.0]
> at
> org.apache.flink.cdc.connectors.paimon.sink.PaimonMetadataApplier.applySchemaChange(PaimonMetadataApplier.java:124)
> ~[flink-cdc-pipeline-connector-paimon-3.3.0.jar:3.3.0]
> at
> org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.applyAndUpdateEvolvedSchemaChange(SchemaCoordinator.java:434)
> ~[flink-cdc-dist-3.3.0.jar:3.3.0]
> at
> org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.applySchemaChange(SchemaCoordinator.java:401)
> ~[flink-cdc-dist-3.3.0.jar:3.3.0]
> at
> org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.lambda$startSchemaChangesEvolve$0(SchemaCoordinator.java:288)
> ~[flink-cdc-dist-3.3.0.jar:3.3.0]
> ... 5 more {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
