I am working with a 3rd party kafka provider and realized that I could use the projected service account token that EKS (AWS's Elastic Kubernetes Service) automatically mounts in a pod with a specified service account (a JWT managed by the AWS OIDC provider <https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html>), to authenticate my services to the Kafka cluster. It seemed like a pretty great solution for short-lived auto-rotated auth tokens (yay security). This is of course after I registered the OIDC provider with our Kafka service provider. I noticed kafka added this OIDC support <https://github.com/apache/kafka/commit/7b379539a5533179b0c86adb352548ac1aa82006> along with a FileTokenRetriever in the Kafka java client some time back. The AccessTokenRetriverFactory <https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/security/oauthbearer/internals/secured/AccessTokenRetrieverFactory.java#L63-L64> creates a FileTokenRetriever if the token endpoint URL is a filepath, e.g. file://some/path/to/token.
So I created a service-account in kubernetes, and EKS mounts an auto-rotated token at /var/run/secrets/ eks.amazonaws.com/serviceaccount/token, which I can then use as the filepath for the token retriever, however, the default implementation only reads the file contents once, so after some time, the token will go stale and as far as I can tell, is never re-read. I then wrote my own implementation of AuthenticateCallbackHandler that is almost the same as the OAuthBearerLoginCallbackHandler but replaces the use of the token retriever factory with my own RefreshingFileTokenRetriever that reads/caches/re-reads the specified file. I am wondering if anyone else has this use-case, basically having workload identities that could have auto-rotated credentials mounted as files that could be periodically read in and used as the OIDC token. This seems like it's already a pattern used by the kubernetes client itself: - k8s client BearerAuthRoundTripper <https://github.com/kubernetes/client-go/blob/da6e2946e51b907f353947a24a91ae91393df734/transport/round_trippers.go#L290-L309> - example of datadog-agent using it <https://github.com/goodspark/datadog-agent/blob/75af92223f9d582e6d8b36fea6624a18a7477160/pkg/util/kubernetes/kubelet/kubelet_client.go#L91-L94>. This is authenticating with the kubernetes API, of course, but as far as I can tell, it's using the same kind of mechanism any OIDC provider/consumer would use, right? Anyway, I am not a whiz when it comes to oauth really. I'm a bit new to it. However, when I figured out I could have a form of short-lived credentials, that my application didn't necessarily have to manage the rotation of, and getting all the benefits of a "workload identity" role, rather than doling out long-lived key/secrets for each service with different scopes, etc. I thought this was amazing and would be a great benefit for the Kafka client. I don't think this would be very difficult to implement. And there is prior art in the form of the kubernetes client itself. Would this be something worth adding as an internal component? Also, am I even thinking about this the right way? Again, I have little experience in OIDC, etc. I have used AWS IAM for a long time though, and want the same role-based workload identity. This seemed to work well so far, but I'm really not even sure it's the "correct" way to do things. I could only imagine the k8s api is pretty well vetted. Anyway, Would love any feedback or linking me to someone with good OIDC knowledge. For now I'll be plumbing in static Vault secrets that aren't rotated regularly 😬 Thanks, - Creed Haymond
