We need to overwrite using
WebIdentityTokenFileCredentialsProviderhttps://github.com/aws/aws-sdk-java-v2/issues/1470#issuecomment-543601232.
otherwise java takes presidency to secret key and access keys than SA
On Saturday, September 25, 2021, 04:37:22 PM EDT, Xiangyu Su
<[email protected]> wrote:
Hi Thomas,did you try to login to EKS node and run some aws command like : aws
s3 ls <backend bucket>?It sounds like EKS issue, but not 100% sure.Best
On Sat, 25 Sept 2021 at 22:12, Ingo Bürk <[email protected]> wrote:
Hi Thomas,
I think you might be looking for this:
https://github.com/apache/flink/pull/16717
BestIngo
On Sat, Sep 25, 2021, 20:46 Thomas Wang <[email protected]> wrote:
Hi,
I'm using the official docker image: apache/flink:1.12.1-scala_2.11-java11
I'm trying to run a Flink job on an EKS cluster. The job is running under a k8s
service account that is tied to an IAM role. If I'm not using s3 as RocksDB
checkpoint backend, everything works just fine. However, when I enabled s3 as
RocksDB checkpoint backend, I got permission denied.
The IAM role tied to the service account has the appropriate permissions to s3.
However the underlying role tied to the EKS node doesn't. After debugging with
AWS support, it looks like the request to s3 was made under the EKS node role,
not the role tied to the service account. Thus the permission denial.
With the same Flink application, I'm also making requests to AWS Secrets
Manager to get some sensitive information and those requests were made
explicitly with AWS Java SDK 2.x bundled in the same application Jar file.
Those requests were made correctly with the IAM role tied to the service
account.
Based on the info above, I suspect Flink may be using an older version of the
AWS SDK that doesn't support assuming an IAM role via an IODC web identity
token file. Please see AWS doc here:
https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-minimum-sdk.html
Could someone help me confirm this bug and maybe have it fixed some time?
Thanks.
Thomas
--
Xiangyu Su
Java Developer
[email protected]
Smaato Inc.
San Francisco - New York - Hamburg - Singapore
www.smaato.com
Germany:
Barcastraße 5
22087 Hamburg
Germany
M 0049(176)43330282
The information contained in this communication may be CONFIDENTIAL and is
intended only for the use of the recipient(s) named above. If you are not the
intended recipient, you are hereby notified that any dissemination,
distribution, or copying of this communication, or any of its contents, is
strictly prohibited. If you have received this communication in error, please
notify the sender and delete/destroy the original message and any copy of it
from your computer or paper files.
