Hi Carlos! Take a look at this project, it's 6 years old but the approach is still valid:
https://github.com/zillow/aws-custom-credential-provider The credential provider gets called each time an S3 or Glue Catalog is accessed, and then you can decide whether to use a cached token or renew. Best, *Pol Santamaria* On Mon, Oct 23, 2023 at 8:08 AM Jörn Franke <jornfra...@gmail.com> wrote: > Can’t you attach the cross account permission to the glue job role? Why > the detour via AssumeRole ? > > Assumerole can make sense if you use an AWS IAM user and STS > authentication, but this would make no sense within AWS for cross-account > access as attaching the permissions to the Glue job role is more secure (no > need for static credentials, automatically renew permissions in shorter > time without any specific configuration in Spark). > > Have you checked with AWS support? > > Am 22.10.2023 um 21:14 schrieb Carlos Aguni <aguni...@gmail.com>: > > > hi all, > > i've a scenario where I need to assume a cross account role to have S3 > bucket access. > > the problem is that this role only allows for 1h time span (no > negotiation). > > that said. > does anyone know a way to tell spark to automatically renew the token > or to dinamically renew the token on each node? > i'm currently using spark on AWS glue. > > wonder what options do I have. > > regards,c. > >
