Re: Error running on Hadoop 2.7

Thanks, in that case it sounds like it is more related to Hadoop classpath mixups, rather than class loading. On Mon, Mar 26, 2018 at 3:03 PM, ashish pok wrote: > Stephan, we are in 1.4.2. > > Thanks, > > -- Ashish > > On Mon, Mar 26, 2018 at 7:38 AM, Stephan Ewen > wrote: > If you are on Flink

Re: Error running on Hadoop 2.7

Stephan, we are in 1.4.2. Thanks, -- Ashish On Mon, Mar 26, 2018 at 7:38 AM, Stephan Ewen wrote: If you are on Flink 1.4.0 or 1.4.1, please check if you accidentally have Hadoop in your application jar. That can mess up things with child-first classloading. 1.4.2 should handle Hadoop pro

Re: Error running on Hadoop 2.7

If you are on Flink 1.4.0 or 1.4.1, please check if you accidentally have Hadoop in your application jar. That can mess up things with child-first classloading. 1.4.2 should handle Hadoop properly in any case. On Sun, Mar 25, 2018 at 3:26 PM, Ashish Pokharel wrote: > Hi Ken, > > Yes - we are on

Re: Error running on Hadoop 2.7

Hi Ken, Yes - we are on 1.4. Thanks for that link - it certainly now explains how things are working :) We currently don’t have HADOOP_CLASSPATH env var setup and “hadoop class path” command basically points to HDP2.6 locations (HDP = Hortonworks Data Platform). Best guess I have for this rig

Re: Error running on Hadoop 2.7

Hi Ashish, Are you using Flink 1.4? If so, what does the “hadoop classpath” command return from the command line where you’re trying to start the job? Asking because I’d run into issues with https://issues.apache.org/jira/browse/FLINK-7477 , wh

Re: Error running on Hadoop 2.7

Hi All, Looks like we are out of the woods for now (so we think) - we went with Hadoop free version and relied on client libraries on edge node. However, I am still not very confident as I started digging into that stack as well and realized what Till pointed out (traces leads to a class that

Re: Error running on Hadoop 2.7

Hi Ashish, Yeah, we also had this problem before. It can be solved by recompiling Flink with HDP version of Hadoop according to instruction here: https://ci.apache.org/projects/flink/flink-docs-release-1.4/start/building.html#vendor-specific-versions Regards, Kien On 3/22/2018 12:25 AM,

Re: Error running on Hadoop 2.7

Hi Ashish, the class `RequestHedgingRMFailoverProxyProvider` was only introduced with Hadoop 2.9.0. My suspicion is thus that you start the client with some Hadoop 2.9.0 dependencies on the class path. Could you please check the logs of the client what's on its class path? Maybe you could also sha

Re: Error running on Hadoop 2.7

Hi Piotrek, At this point we are simply trying to start a YARN session.  BTW, we are on Hortonworks HDP 2.6 which is on 2.7 Hadoop if anyone has experienced similar issues.  We actually pulled 2.6 binaries for the heck of it and ran into same issues.  I guess we are left with getting non-hadoop bi

Re: Error running on Hadoop 2.7

Hi, > Does some simple word count example works on the cluster after the upgrade? If not, maybe your job is pulling some dependency that’s causing this version conflict? Piotrek > On 21 Mar 2018, at 16:52, ashish pok wrote: > > Hi Piotrek, > > Yes, this is a brand new Prod environment. 2.6

Re: Error running on Hadoop 2.7

Hi Piotrek, Yes, this is a brand new Prod environment. 2.6 was in our lab. Thanks, -- Ashish On Wed, Mar 21, 2018 at 11:39 AM, Piotr Nowojski wrote: Hi, Have you replaced all of your old Flink binaries with freshly downloaded Hadoop 2.7 versions? Are you sure that something hasn't mix in

Re: Error running on Hadoop 2.7

Hi, Have you replaced all of your old Flink binaries with freshly downloaded Hadoop 2.7 versions? Are you sure that something hasn't mix in the process? Does some simple word count example works on the cluster after the upgrade? Piotrek > On 21 Mar 20

Error running on Hadoop 2.7

Hi All, We ran into a roadblock in our new Hadoop environment, migrating from 2.6 to 2.7. It was supposed to be an easy lift to get a YARN session but doesnt seem like :) We definitely are using 2.7 binaries but it looks like there is a call here to a private methos which screams runtime incompa