Re: User jar is present in the flink job manager's class path

2018-10-11 Thread yinhua.dai
Hi Gary, Yes you are right, we are using the attach mode. I will try to put my jar to flink/lib to get around with the issue. Thanks. I will open a jira for the discrepancy for flink 1.3 and 1.5, thanks a lot. -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: User jar is present in the flink job manager's class path

2018-10-11 Thread Gary Yao
Hi, Could it be that you are submitting the job in attached mode, i.e., without -d parameter? In the "job cluster attached mode", we actually start a Flink session cluster (and stop it again from the CLI) [1]. Therefore, in attached mode, the config option "yarn.per-job-cluster.include-user-jar" i

Re: User jar is present in the flink job manager's class path

2018-10-11 Thread yinhua.dai
Meanwhile, I can see below code in flink 1.5 public static final ConfigOption CLASSPATH_INCLUDE_USER_JAR = key("yarn.per-job-cluster.include-user-jar") .defaultValue("ORDER") .withDescription("Defines whether user-jars are included i

Re: User jar is present in the flink job manager's class path

2018-10-11 Thread Timo Walther
Yes, you are right. I was not aware that the resolution order depends on the cluster deployment. I will loop in Gary (in CC) that might know about such a YARN setup. Regards, Timo Am 11.10.18 um 15:47 schrieb yinhua.dai: Hi Timo, I didn't tried to configure the classloader order, according t

Re: User jar is present in the flink job manager's class path

2018-10-11 Thread yinhua.dai
Hi Timo, I didn't tried to configure the classloader order, according to the document, it should only be needed for yarn-session mode, right? I can see the ship files(-yt /path/dir/) is present in job manager's class path, so maybe I should put my uber jar in the -yt path so that it will be shipp

Re: User jar is present in the flink job manager's class path

2018-10-11 Thread Timo Walther
Hi, did you try to change the classloading strategy? Maybe this problem could be fixed by configuring the ClassLoader resolution order [1]. Regards, Timo [1] https://ci.apache.org/projects/flink/flink-docs-release-1.5/monitoring/debugging_classloading.html Am 11.10.18 um 10:49 schrieb yinh