hi Chirag, Flink now supports 2 ways to have TGT which is a Kerberos ticket and has nothing to do with the "until 7 days renewable" HDFS TGS ticket (with default config).
* Keytab: if one mounts a keytab for at least the JobManager pod then it can create TGT infinitely (or until the user's password is not changed). * Kerberos ticket cache: Such case the user's responsibility to create/renew TGT for example with kinit command. If the user is not creating or forgetting to re-obtain the TGT then the Flink workload is going to fail. When the JM has a valid TGT then it can ask token providers to obtain a service specific token TGS. In case of HDFS with default config this can be renewed for 7 days within every 24 hours. When the HDFS side configured 7days is gone then TGS must be re-obtained with a valid TGT. So this is basically how it works. > what should be the KDC Don't understand the question at all. Any kind of Kerberos usage is starting with a "create a KDC server in your environment". That sever must be set. > I understand that Hadoop automatically renews the TGT using keytab. Most of the time it does but not all the times. In Flink's delegation token model it doesn't really matter what Hadoop does, Flink does everything which is required when we speak about token based authentication. > Does this require us to remount the keytab file again? That said until the user's password is not changed or the user is not deactivated no remount is needed. When we speak about keytab in an oversimplified and not exact way one can consider it as a plaintext password file plus some magic. As a general advise please have a look at the following doc: https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/security/security-delegation-token/ G On Tue, Sep 5, 2023 at 1:31 PM Chirag Dewan via user <user@flink.apache.org> wrote: > Hi, > > I am trying to use the FileSource to collect files from HDFS. The HDFS > cluster is secured and has Kerberos enabled. > > My Flink cluster runs on Kubernetes (not using the Flink operator) with 2 > Job Managers in HA and 3 Task Managers. I wanted to understand the correct > way to configure the keytab files for this cluster. > > Should I just mount the keytab file in all pods and use the > "security.kerberos.login.keytab" config? > > For such a setup, what should be the KDC and how do we login to the KDC > using kinit? > > I understand that Hadoop automatically renews the TGT using keytab. But > this should only be possible for the renewable lifetime of the ticket > (typically 7 days). Does this require us to remount the keytab file again? > > Any insights will be helpful here. Thanks > >
