Re: Unable to run flink job in dataproc cluster with jobmanager provided

2020-06-22 Thread Chesnay Schepler
Is your user-jar packaging and relocating Flink classes? If so, then your job actually operate against the classes provided by the cluster, which, well, just wouldn't work. On 18/06/2020 09:34, Sourabh Mehta wrote: Hi , application is using 1.10.0 but cluster is setup on 1.9.0. Yes I do have

Re: Unable to run flink job in dataproc cluster with jobmanager provided

2020-06-18 Thread Sourabh Mehta
Hi , application is using 1.10.0 but cluster is setup on 1.9.0. Yes I do have access. please find below starting logs from cluster 2020-06-17 11:28:18,989 INFO org.apache.shaded.flink.table.module.ModuleManager- Got FunctionDefinition equals from module core 2020-06-17 11:28:20,538

Re: Unable to run flink job in dataproc cluster with jobmanager provided

2020-06-17 Thread Till Rohrmann
Hi Sourabh, do you have access to the cluster logs? They could be helpful for debugging the problem. Which version of Flink are you using? Cheers, Till On Wed, Jun 17, 2020 at 7:39 PM Sourabh Mehta wrote: > No, I am not. > > On Wed, 17 Jun 2020 at 10:48 PM, Chesnay Schepler > wrote: > >> Are

Re: Unable to run flink job in dataproc cluster with jobmanager provided

2020-06-17 Thread Sourabh Mehta
No, I am not. On Wed, 17 Jun 2020 at 10:48 PM, Chesnay Schepler wrote: > Are you by any chance creating a local environment via > (Stream)ExecutionEnvironment#createLocalEnvironment? > > On 17/06/2020 17:05, Sourabh Mehta wrote: > > Hi Team, > > I'm exploring flink for one of my use case, I'm f

Re: Unable to run flink job in dataproc cluster with jobmanager provided

2020-06-17 Thread Chesnay Schepler
Are you by any chance creating a local environment via (Stream)ExecutionEnvironment#createLocalEnvironment? On 17/06/2020 17:05, Sourabh Mehta wrote: Hi Team, I'm  exploring flink for one of my use case, I'm facing some issues while running a flink job in cluster mode. Below are the steps I

Unable to run flink job in dataproc cluster with jobmanager provided

2020-06-17 Thread Sourabh Mehta
Hi Team, I'm exploring flink for one of my use case, I'm facing some issues while running a flink job in cluster mode. Below are the steps I followed to setup and run job in cluster mode : 1. Setup flink on google cloud dataproc using https://github.com/GoogleCloudDataproc/initialization-actions/