Re: A classloading question about submitting Flink jobs on Yarn-Perjob Mode

[hjw]

Hi,


The yarn.classpath.include-user-jar parameter is shown as 
yarn.per-job-cluster.include-user-jar parameter in Flink 1.14. 
I have tried DISABLED??FIRST??LAST??ORDER .But the error still happened.

Best,
Hjw


 




------------------ Original ------------------
From:                                                                           
                                             "zhanghao.chen"                    
                                                                
<zhanghao.c...@outlook.com&gt;;
Date:&nbsp;Tue, Sep 13, 2022 09:42 AM
To:&nbsp;"hjw"<1010445...@qq.com&gt;;"user"<user@flink.apache.org&gt;;

Subject:&nbsp;Re: A classloading question about  submitting Flink jobs on 
Yarn-Perjob Mode



  Hi,
  
 
  Did you set any additional classloading-related configs (esp. the 
yarn.classpath.include-user-jar parameter)?
   
 
   
 
  Best,
 Zhanghao Chen
 
 
 
 
 
 
 
 From: hjw <1010445...@qq.com&gt;
 Sent: Tuesday, September 13, 2022 1:58
 To: user <user@flink.apache.org&gt;
 Subject: A classloading question about submitting Flink jobs on Yarn-Perjob 
Mode &nbsp;
 
  When I submit a job to yarn on Yarn-perjob Mode.An error occurred during the 
client-side generation of the Jobgraph submitd to yarn.
 
 
 Error:
 java.lang.NoClassDefFoundError??org/apache/orc/PhysicalWriter
 
 
 Because the cluster is public, there is already related 
flink-orc_2.12-1.14.0-csa1.6.1.0.jar package in the /opt/flink/lib directory.
 
 
 However ,this class is included by my jar. This class is provided by orc-core 
package. I have packaged it in my jar.
 
 
 After my attempts, the following measures can solve my problem.
 1.remove the related flink-orc_2.12-1.14.0-csa1.6.1.0.jar from /opt/flink/lib 
directory and packaged it to my jar.
 2.put the orc-core to /opt/flink/lib directory.
 
 
 However, I would like to know why an error occurs when placing the 
flink-orc_2.12-1.14.0-csa1.6.1.0.jar package in the /opt/flink/lib directory 
and packaging orc-core into the my jar.
 
 
 
 
 Env??
 Flink version: flink 1.14.0
 
 
  
  Best,
 Hjw
 
 
 &nbsp;