Hi, Thanks for your feedback. So I guess I'll have to talk to the security guys about having special kerberos ticket expiry times for these types of jobs.
Niels Basjes On Fri, Oct 23, 2015 at 11:45 AM, Maximilian Michels <m...@apache.org> wrote: > Hi Niels, > > Thank you for your question. Flink relies entirely on the Kerberos > support of Hadoop. So your question could also be rephrased to "Does > Hadoop support long-term authentication using Kerberos?". And the > answer is: Yes! > > While Hadoop uses Kerberos tickets to authenticate users with services > initially, the authentication process continues differently > afterwards. Instead of saving the ticket to authenticate on a later > access, Hadoop creates its own security tockens (DelegationToken) that > it passes around. These are authenticated to Kerberos periodically. To > my knowledge, the tokens have a life span identical to the Kerberos > ticket maximum life span. So be sure to set the maximum life span very > high for long streaming jobs. The renewal time, on the other hand, is > not important because Hadoop abstracts this away using its own > security tockens. > > I'm afraid there is not Kerberos how-to yet. If you are on Yarn, then > it is sufficient to authenticate the client with Kerberos. On a Flink > standalone cluster you need to ensure that, initially, all nodes are > authenticated with Kerberos using the kinit tool. > > Feel free to ask if you have more questions and let us know about any > difficulties. > > Best regards, > Max > > > > On Thu, Oct 22, 2015 at 2:06 PM, Niels Basjes <ni...@basjes.nl> wrote: > > Hi, > > > > I want to write a long running (i.e. never stop it) streaming flink > > application on a kerberos secured Hadoop/Yarn cluster. My application > needs > > to do things with files on HDFS and HBase tables on that cluster so > having > > the correct kerberos tickets is very important. The stream is to be > ingested > > from Kafka. > > > > One of the things with Kerberos is that the tickets expire after a > > predetermined time. My knowledge about kerberos is very limited so I hope > > you guys can help me. > > > > My question is actually quite simple: Is there an howto somewhere on how > to > > correctly run a long running flink application with kerberos that > includes a > > solution for the kerberos ticket timeout ? > > > > Thanks > > > > Niels Basjes > -- Best regards / Met vriendelijke groeten, Niels Basjes
