Does Spark support role-based authentication and access to Amazon S3? (Kubernetes cluster deployment)

Hello Team, Does Spark support role-based authentication and access to Amazon S3 for Kubernetes deployment? *Note: we have deployed our spark application in the Kubernetes cluster.* Below are the Hadoop-AWS dependencies we are using: org.apache.hadoop hadoop-aws 3.3.4 We are u

Issue with Spark Session Initialization in Kubernetes Deployment

Hello Team, I am currently working on initializing a Spark session using Spark Structure Streaming within a Kubernetes deployment managed by the Spark Operator. During the initialization process, I encountered an error message indicating the necessity to set a master URL: *"Caused by: org.apache.