peiyu created FLINK-40043:
-----------------------------
Summary: Paimon pipeline job couldn’t increase job parallelism
Key: FLINK-40043
URL: https://issues.apache.org/jira/browse/FLINK-40043
Project: Flink
Issue Type: Bug
Components: Flink CDC
Affects Versions: cdc-3.6.0
Environment: flink 1.20.3
flinkcdc 3.6.0
Reporter: peiyu
Fixed the issue where Paimon pipeline job couldn’t increase job parallelism.
Reproduction method:
1. Stop the CDC job using a savepoint
2. Increase the parallelism of the task
3. Start the job using a savepoint
Error message:
```
java.util.NoSuchElementException
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:1000)
at
org.apache.flink.cdc.connectors.paimon.sink.v2.PaimonSink.restoreWriter(PaimonSink.java:101)
at
org.apache.flink.streaming.runtime.operators.sink.StatefulSinkWriterStateHandler.createWriter(StatefulSinkWriterStateHandler.java:120)
at
org.apache.flink.streaming.runtime.operators.sink.SinkWriterOperator.initializeState(SinkWriterOperator.java:153)
at
org.apache.flink.cdc.runtime.operators.sink.DataSinkWriterOperator.initializeState(DataSinkWriterOperator.java:132)
at
org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.initializeOperatorState(StreamOperatorStateHandler.java:147)
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:294)
at
org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:106)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.restoreStateAndGates(StreamTask.java:858)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$restoreInternal$5(StreamTask.java:812)
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:812)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:771)
at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:970)
at
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:939)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:763)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575)
at java.base/java.lang.Thread.run(Thread.java:829)
```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
