Re: [External] Re: Flink and S3 AWS keys rotation

2019-02-07 Thread Antonio Verardi
Hi Bruno, The problem with such a solution would be that those permissions will apply to any application running on the Kubernetes cluster, not only to Flink. Sharing resources with other applications is one of the cool things about Kubernetes and it would be ideal not to lose such a property. Tha

Re: Flink and S3 AWS keys rotation

2019-02-07 Thread Bruno Aranda
Hi, You can give specific IAM instance roles to the instances running Flink. This way you never expose access keys anywhere. As the docs say, that is the recommended way (and not just for Flink, but for any service you want to use, never set it up with AWS credentials in config). IAM will transpar

Re: Flink and S3 AWS keys rotation

2019-02-07 Thread Kostas Kloudas
Hi Antonio, I am cc'ing Till who may have something to say on this. Cheers, Kostas On Thu, Feb 7, 2019 at 1:32 PM Antonio Verardi wrote: > Hi there, > > I'm trying out to run Flink on Kubernetes and I run into a problem with > the way Flink sets up AWS credentials to talk with S3 and the way

Flink and S3 AWS keys rotation

2019-02-07 Thread Antonio Verardi
Hi there, I'm trying out to run Flink on Kubernetes and I run into a problem with the way Flink sets up AWS credentials to talk with S3 and the way we manage AWS secrets in my company. To give permissions to Flink I am using AWS keys embedded in flink.conf, as per https://ci.apache.org/projects/f