Hi Eron, Thank you for your feedback! Indeed, we have seen in the past, that Hadoop's Delegation Tokens are not meant to renewed over a long period. Plus, they have a number of subtle bugs in older versions that sometimes prevent renewal.
What you suggest, sounds like a good approach to me. It would basically mean that we handle have our own renewal system and do not rely on Hadoop's token renewal system. Improving long-running Kerberos jobs: https://issues.apache.org/jira/browse/FLINK-3670 Kafka Kerberos support JIRA: https://issues.apache.org/jira/browse/FLINK-3239 Thanks, Max On Sat, Mar 26, 2016 at 12:18 AM, Eron Wright <ewri...@live.com> wrote: > (fixed bad formatting) > > Hi, > Given the other thread about per-job Kerberos identity, now's a good time to > discuss some problems with the current delegation-token approach, since the > answer could bear on the per-job enhancement. > > I see two problems: > > 1. Delegation tokens expire. For a continuous streaming job to survive, the > original keytab is needed to re-authenticate. Spark Streaming solved this > problem with `--keytab` on spark-submit (see AMDelegationTokenRenewer.scala). > > 2. Kafka doesn't support delegation tokens yet (see KIP-48 and KAFKA-1696). > > Thoughts? Thanks! > - Eron Wright > > > >
