Hi, the fsGroup setting should match the id Spark is running at. When building from source, that id is 185, and you can use "docker inspect <image-name>" to double-check.
On Wed, Feb 10, 2021 at 11:43 AM Rishabh Jain <rishabh.j...@thoughtworks.com> wrote: > Hi, > > I tried doing what Vladimir suggested. But no luck there either. My guess > is that it has something to do with securityContext.fsGroup. I am trying to > pass yaml file path along with spark submit command. My yaml file content > is > ``` > > apiVersion: v1 > > kind: Pod > > spec: > > securityContext: > > fsGroup: 65534 > > serviceAccount: <service accoun> > > serviceAccountName: <service account name> > > ``` > > > Is there anything wrong with this yaml file? > > > ~ > *Thanks,* > > Rishabh Jain > Application Developer > Email rishabh.j...@thoughtworks.com > Telephone +91 6264277897 <+91+626+427+7897> > [image: ThoughtWorks] > <http://www.thoughtworks.com/?utm_campaign=prajwal-boloor-signature&utm_medium=email&utm_source=thoughtworks-email-signature-generator> > > > > > On Tue, Feb 9, 2021 at 10:44 PM Vladimir Prus <vladimir.p...@gmail.com> > wrote: > >> >> >> On 9 Feb 2021, at 19:46, Rishabh Jain <rishabh.j...@thoughtworks.com> >> wrote: >> >> Hi, >> >> We are trying to access S3 from spark job running on EKS cluster pod. I >> have a service account that has an IAM role attached with full S3 >> permission. We are using DefaultCredentialsProviderChain. But still we are >> getting 403 Forbidden from S3. >> >> >> It’s hard to say without any information, but some things you might want >> to double-check >> >> - Make sure the Spark job is using sufficiently new AWS SDK, so that IAM >> for service account is supported >> - Modify your job to print the effective role, e.g. >> >> val stsClient = >> AWSSecurityTokenServiceClientBuilder.standard().build(); >> val request = new GetCallerIdentityRequest() >> val identity = stsClient.getCallerIdentity(request) >> println(identity.getArn()) >> >> - If the above does not print the expected role, verify that the pods >> actually have the right service account, and >> that AWS_ROLE_ARN/AWS_WEB_IDENTITY_TOKEN_FILE variables are set on the >> pod, and that >> the assume policy for the role does allow EKS to assume that role. >> - If the above prints the expected role, then 403 error means you did not >> setup IAM policies on your role/bucket. >> >> >> Is there anything wrong with our approach? >> >> Generally speaking, IAM for service accounts in EKS + Spark works, it's >> just there's a lot of things that can go wrong the first time you do it. >> >> >> HTH, >> > -- Vladimir Prus http://vladimirprus.com
