Hi,
The yarn.classpath.include-user-jar parameter is shown as
yarn.per-job-cluster.include-user-jar parameter in Flink 1.14.
I have tried DISABLED??FIRST??LAST??ORDER .But the error still happened.
Best,
Hjw
------------------ Original ------------------
From:
"zhanghao.chen"
<[email protected]>;
Date: Tue, Sep 13, 2022 09:42 AM
To: "hjw"<[email protected]>;"user"<[email protected]>;
Subject: Re: A classloading question about submitting Flink jobs on
Yarn-Perjob Mode
Hi,
Did you set any additional classloading-related configs (esp. the
yarn.classpath.include-user-jar parameter)?
Best,
Zhanghao Chen
From: hjw <[email protected]>
Sent: Tuesday, September 13, 2022 1:58
To: user <[email protected]>
Subject: A classloading question about submitting Flink jobs on Yarn-Perjob
Mode
When I submit a job to yarn on Yarn-perjob Mode.An error occurred during the
client-side generation of the Jobgraph submitd to yarn.
Error:
java.lang.NoClassDefFoundError??org/apache/orc/PhysicalWriter
Because the cluster is public, there is already related
flink-orc_2.12-1.14.0-csa1.6.1.0.jar package in the /opt/flink/lib directory.
However ,this class is included by my jar. This class is provided by orc-core
package. I have packaged it in my jar.
After my attempts, the following measures can solve my problem.
1.remove the related flink-orc_2.12-1.14.0-csa1.6.1.0.jar from /opt/flink/lib
directory and packaged it to my jar.
2.put the orc-core to /opt/flink/lib directory.
However, I would like to know why an error occurs when placing the
flink-orc_2.12-1.14.0-csa1.6.1.0.jar package in the /opt/flink/lib directory
and packaging orc-core into the my jar.
Env??
Flink version: flink 1.14.0
Best,
Hjw
