Hi Sri, I logged a cloudera ticket, as you recommended, and got help from their support team, and was able to get my application running.
We had to “kinit” inside the shell action using a keytab in the following format: “kinit primary/instance@REALM -kt primary.keytab” The keytab file had to be added to the lib folder the workflow root. The shell action localizes the keytab to the container’s current working directory and then can be used for knit in the oozie shell action. Thanks for your help on this issue! Best, Rajat Srivastava From: sri hari kali charan Tummala <kali.tumm...@gmail.com> Date: Monday, October 7, 2019 at 10:41 AM To: "Srivastava,Rajat" <rajat.srivast...@cerner.com> Subject: Re: Flink using Oozie in Kerberized cluster echo "password" | /usr/bin/kinit use...@activedirectory.domain.net<mailto:use...@activedirectory.domain.net> this is something we use to do in your first step in our Oozie workflow and then run the rest of the workflow, as I said Cloudera might help. On Mon, Oct 7, 2019 at 9:10 AM sri hari kali charan Tummala <kali.tumm...@gmail.com<mailto:kali.tumm...@gmail.com>> wrote: please raise a ticket with cloudera its kerberos issue. On Sun, Oct 6, 2019 at 11:26 PM Srivastava,Rajat <rajat.srivast...@cerner.com<mailto:rajat.srivast...@cerner.com>> wrote: Thanks for your response! We are doing something very similar, building a fat jar, uploading it to HDFS, and using shell action to run the application. The issue we are having however is running the application on a Kerberized cluster in yarn-cluster mode. We get the following error: ERROR org.apache.flink.yarn.AbstractYarnClusterDescriptor - Hadoop security with Kerberos is enabled but the login user does not have Kerberos credentials Best, Rajat Srivastava From: sri hari kali charan Tummala <kali.tumm...@gmail.com<mailto:kali.tumm...@gmail.com>> Date: Saturday, October 5, 2019 at 1:47 PM To: "Srivastava,Rajat" <rajat.srivast...@cerner.com<mailto:rajat.srivast...@cerner.com>> Subject: Re: Flink using Oozie in Kerberized cluster https://youtu.be/Q6gEVidrGNE<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fyoutu.be%2FQ6gEVidrGNE&data=02%7C01%7CRajat.Srivastava%40cerner.com%7C82b5fb45850645f049ee08d74b3cdf92%7Cfbc493a80d244454a815f4ca58e8c09d%7C0%7C0%7C637060597146084177&sdata=HS55V4vQgh8%2BFoGCeVitWx%2BzyF3sbIqz%2FVyNxDPYNZw%3D&reserved=0> I usually build a fat jar and call it as shell action as explained in the video I also upload the jar to hdfs. That’s all really , not sure about Apache beam pipeline. Thanks Sri On Friday, October 4, 2019, Srivastava,Rajat <rajat.srivast...@cerner.com<mailto:rajat.srivast...@cerner.com>> wrote: Moving out of Cloudera is not an option for us. By bounded flink, I actually meant by running a bounded beam pipeline on a flink runner. Best, Rajat Srivastava From: sri hari kali charan Tummala <kali.tumm...@gmail.com<mailto:kali.tumm...@gmail.com>> Date: Friday, October 4, 2019 at 11:41 AM To: "Srivastava,Rajat" <rajat.srivast...@cerner.com<mailto:rajat.srivast...@cerner.com>> Subject: Re: Flink using Oozie in Kerberized cluster AWS has kinesis data analytics which provisions manged Flink cluster and runs the job for you think go moving out of Cloudera, check screen shot attached. Ozzie & Cloudera:- what is bounded Flink Application mean in your email? On Fri, Oct 4, 2019 at 10:23 AM Srivastava,Rajat <rajat.srivast...@cerner.com<mailto:rajat.srivast...@cerner.com>> wrote: It’s on a Cloudera managed cluster. Best, Rajat Srivastava From: sri hari kali charan Tummala <kali.tumm...@gmail.com<mailto:kali.tumm...@gmail.com>> Date: Friday, October 4, 2019 at 7:39 AM To: "Srivastava,Rajat" <rajat.srivast...@cerner.com<mailto:rajat.srivast...@cerner.com>> Subject: Re: Flink using Oozie in Kerberized cluster is this on AWS or AWS EMR or Cloudera ? On Thu, Oct 3, 2019 at 3:51 PM Srivastava,Rajat <rajat.srivast...@cerner.com<mailto:rajat.srivast...@cerner.com>> wrote: Hi, Has someone tried to run bounded Flink application using Oozie in a Kerberized cluster ? What else can we use to schedule bounded flink jobs. Any thoughts or ideas will be appreciated! Flink version: 1.8.0 Flink-runner version: 2.14.0 Best, Rajat Srivastava Software Engineer | HealtheIntent Dev rajat.srivast...@cerner.com<mailto:rajat.srivast...@cerner.com> | (816) 906-4958 direct Cerner Corporation | www.cerner.com<http://www.cerner.com/> CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024. -- Thanks & Regards Sri Tummala -- Thanks & Regards Sri Tummala -- Thanks & Regards Sri Tummala -- Thanks & Regards Sri Tummala -- Thanks & Regards Sri Tummala
