Thanks Greg, this is a really helpful reply.
>Any kind of Kerberos usage is starting with a "create a KDC server in your
>environment". That sever must be set.
When I say that I am kind of referring to Windows users with inbuild KDC and
AD. That would require kinit for the AS. I was wondering how that would work
inside the pod.
My understanding of Kerberos isnt sound yet, so it maybe a very stupid question
:)
This link is for Flink 1.19. Are the DTs also applicable in Flink 1.16?
Thanks
On Tuesday, 5 September, 2023 at 07:15:07 pm IST, Gabor Somogyi
<gabor.g.somo...@gmail.com> wrote:
hi Chirag,
Flink now supports 2 ways to have TGT which is a Kerberos ticket and has
nothing to do with the "until 7 days renewable" HDFS TGS ticket (with default
config).
* Keytab: if one mounts a keytab for at least the JobManager pod then it can
create TGT infinitely (or until the user's password is not changed).* Kerberos
ticket cache: Such case the user's responsibility to create/renew TGT for
example with kinit command. If the user is not creating or forgetting to
re-obtain the TGT then the Flink workload is going to fail.
When the JM has a valid TGT then it can ask token providers to obtain a service
specific token TGS. In case of HDFS with default config this can be renewed for
7 days within every 24 hours.When the HDFS side configured 7days is gone then
TGS must be re-obtained with a valid TGT.
So this is basically how it works.
> what should be the KDCDon't understand the question at all. Any kind of
>Kerberos usage is starting with a "create a KDC server in your environment".
>That sever must be set.
> I understand that Hadoop automatically renews the TGT using keytab.Most of
>the time it does but not all the times. In Flink's delegation token model it
>doesn't really matter what Hadoop does, Flink does everything which is
>required when we speak about token based authentication.
> Does this require us to remount the keytab file again?That said until the
>user's password is not changed or the user is not deactivated no remount is
>needed. When we speak about keytab in an oversimplified and not exact way one
>can consider it as a plaintext password file plus some magic.
As a general advise please have a look at the following doc:
https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/security/security-delegation-token/
G
On Tue, Sep 5, 2023 at 1:31 PM Chirag Dewan via user <user@flink.apache.org>
wrote:
Hi,
I am trying to use the FileSource to collect files from HDFS. The HDFS cluster
is secured and has Kerberos enabled.
My Flink cluster runs on Kubernetes (not using the Flink operator) with 2 Job
Managers in HA and 3 Task Managers. I wanted to understand the correct way to
configure the keytab files for this cluster.
Should I just mount the keytab file in all pods and use the
"security.kerberos.login.keytab" config?
For such a setup, what should be the KDC and how do we login to the KDC using
kinit?
I understand that Hadoop automatically renews the TGT using keytab. But this
should only be possible for the renewable lifetime of the ticket (typically 7
days). Does this require us to remount the keytab file again?
Any insights will be helpful here. Thanks
