Re: Questions regarding classpath loading order in YarnClusterDescriptor

2022-06-05 Thread Geng Biao
o Geng From: Leon Xu Date: Sunday, June 5, 2022 at 4:04 PM To: Biao Geng Cc: user Subject: Re: Questions regarding classpath loading order in YarnClusterDescriptor Hi Biao, I really appreciate your thorough answers. And yes for now I took the workaround by manipulating the directory names. To

Re: Questions regarding classpath loading order in YarnClusterDescriptor

2022-06-05 Thread Leon Xu
Hi Biao, I really appreciate your thorough answers. And yes for now I took the workaround by manipulating the directory names. To follow up with one more question if you don't mind: What is the recommended way of managing plugins in YarnClusterDescriptor? Currently I am placing the plugins (e.g. f

Re: Questions regarding classpath loading order in YarnClusterDescriptor

2022-06-04 Thread Biao Geng
Hi Leon, For your question1, in the classpath, there are 2 types of jars: user jars and flink system jars(i.e. jars in flink/lib). System jars are sorted alphabetically. For user jars, there are 3 choices to add user jars in the final classpath: ORDER, FIRST, LAST(See the doc

Questions regarding classpath loading order in YarnClusterDescriptor

2022-06-04 Thread Leon Xu
Hi Flink Community, We are building on top of *org.apache.flink.yarn.YarnClusterDescriptor *to submit a flink application from Java code to YARN cluster, in the application mode. We are setting the classpath as the value of *the yarn.provided.lib.dirs *property under the yarn configuration. By p