I also noticed that Flink s3-hadoop plugin has Hadoop common dependency. I'
trying this.
>From the logs, the plugin is enabled:
Enabling required built-in plugins
Linking flink-s3-fs-hadoop-1.12-SNAPSHOT.jar to plugin directory
Successfully enabled flink-s3-fs-hadoop-1.12-SNAPSHOT.jar
But I a
As I know, flink-shaded-hadoop is not officially supported since Flink 1.11
(https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html).
Anyway, I installed Hadoop common package into the docker images to make Flink
happy. I marked the hadoop dependencies in the i
I prefer using flink bundled hadoop, such as
https://repo.maven.apache.org/maven2/org/apache/flink/flink-shaded-hadoop-2-uber/2.8.3-10.0/flink-shaded-hadoop-2-uber-2.8.3-10.0.jar.
May help.
L. C. Hsieh 于2021年8月22日周日 上午1:40写道：
>
> BTW, I checked dependency tree, the flink-iceberg demo only has on
BTW, I checked dependency tree, the flink-iceberg demo only has one Hadoop
common dependency. So I'm not sure why Flink throws such exception. Based on
Flink doc, I suppose that Flink binary doesn't include Hadoop dependencies,
right?
Based on the exception, looks like when FlinkCatalogFactor
Thanks for replying.
I'm using Flink 1.12.x. And I think Iceberg 0.12 uses Flink 1.12 actually.
Once I upgraded the Iceberg from 0.11.0 to 0.12.0 for the Java application. I
got new exception as below:
java.lang.LinkageError: loader constraint violation: when resolving method
"org.apache.fli
Iceberg v0.11 or v0.12 not capable with flink v1.13.x.
L. C. Hsieh 于2021年8月21日周六 下午3:52写道：
> Hi, I'm testing using Flink to write Iceberg table. I run Flink native K8S
> cluster locally and submit a simple Java program that writes out Iceberg
> table (https://github.com/spancer/flink-iceberg-dem
Hi, I'm testing using Flink to write Iceberg table. I run Flink native K8S
cluster locally and submit a simple Java program that writes out Iceberg table
(https://github.com/spancer/flink-iceberg-demo). But got an exception:
java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration at
