Hongshun Wang created FLINK-38496:
-------------------------------------
Summary: Icerberg yaml sink need extra hadoop jar
Key: FLINK-38496
URL: https://issues.apache.org/jira/browse/FLINK-38496
Project: Flink
Issue Type: Improvement
Components: Documentation, Flink CDC
Affects Versions: cdc-3.5.0
Reporter: Hongshun Wang
Fix For: cdc-3.5.0
Currently, when we use Icerberg yaml sink, error will be thrown:
{code:java}
2025-10-10 19:30:01java.lang.NoClassDefFoundError:
org/apache/hadoop/conf/Configuration at
org.apache.flink.cdc.connectors.iceberg.sink.v2.IcebergCommitter.<init>(IcebergCommitter.java:73)
at
org.apache.flink.cdc.connectors.iceberg.sink.v2.IcebergSink.createCommitter(IcebergSink.java:85)
at
org.apache.flink.streaming.runtime.operators.sink.CommitterOperatorFactory.lambda$createStreamOperator$0(CommitterOperatorFactory.java:65)
at
org.apache.flink.streaming.runtime.operators.sink.CommitterOperator.initializeState(CommitterOperator.java:121)
at
org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.initializeOperatorState(StreamOperatorStateHandler.java:147)
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:294)
at
org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:106)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.restoreStateAndGates(StreamTask.java:858)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$restoreInternal$5(StreamTask.java:812)
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:812)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:771)
at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:970)
at
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:939) at
org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:763) at
org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) at
java.base/java.lang.Thread.run(Thread.java:834) {code}
problem will be solve if
[https://mvnrepository.com/artifact/org.apache.flink/flink-shaded-hadoop-2-uber/2.8.3-10.0]
is added into lib.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
