hi Chirag,

Flink now supports 2 ways to have TGT which is a Kerberos ticket and has
nothing to do with the "until 7 days renewable" HDFS TGS ticket (with
default config).

* Keytab: if one mounts a keytab for at least the JobManager pod then it
can create TGT infinitely (or until the user's password is not changed).
* Kerberos ticket cache: Such case the user's responsibility to
create/renew TGT for example with kinit command. If the user is not
creating or forgetting to re-obtain the TGT then the Flink workload is
going to fail.

When the JM has a valid TGT then it can ask token providers to obtain a
service specific token TGS. In case of HDFS with default config this can be
renewed for 7 days within every 24 hours.
When the HDFS side configured 7days is gone then TGS must be re-obtained
with a valid TGT.

So this is basically how it works.

> what should be the KDC
Don't understand the question at all. Any kind of Kerberos usage is
starting with a "create a KDC server in your environment". That sever must
be set.

> I understand that Hadoop automatically renews the TGT using keytab.
Most of the time it does but not all the times. In Flink's delegation token
model it doesn't really matter what Hadoop does, Flink does everything
which is required when we speak about token based authentication.

> Does this require us to remount the keytab file again?
That said until the user's password is not changed or the user is not
deactivated no remount is needed. When we speak about keytab in an
oversimplified and not exact way one can consider it as a plaintext
password file plus some magic.

As a general advise please have a look at the following doc:
https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/security/security-delegation-token/

G


On Tue, Sep 5, 2023 at 1:31 PM Chirag Dewan via user <user@flink.apache.org>
wrote:

> Hi,
>
> I am trying to use the FileSource to collect files from HDFS. The HDFS
> cluster is secured and has Kerberos enabled.
>
> My Flink cluster runs on Kubernetes (not using the Flink operator) with 2
> Job Managers in HA and 3 Task Managers. I wanted to understand the correct
> way to configure the keytab files for this cluster.
>
> Should I just mount the keytab file in all pods and use the
> "security.kerberos.login.keytab" config?
>
> For such a setup, what should be the KDC and how do we login to the KDC
> using kinit?
>
> I understand that Hadoop automatically renews the TGT using keytab. But
> this should only be possible for the renewable lifetime of the ticket
> (typically 7 days). Does this require us to remount the keytab file again?
>
> Any insights will be helpful here. Thanks
>
>

Reply via email to