Spark on k8s issues with s3a committer dependencies or config?

2022-03-19 Thread Prasad Paravatha
reading from s3 works, I am getting error 403 access denied while writing to the KMS enabled bucket. I am wondering if I am missing some dependency jars or client configuration properties. I would Appreciate your help if someone can give me a few pointers on this. Regards, Prasad Paravatha

Re: One click to run Spark on Kubernetes

2022-02-22 Thread Prasad Paravatha
ngress, and Spark Operator. Then you will >>> be able to use curl or a CLI tool to submit Spark application. After the >>> deployment, you could also install Uber Remote Shuffle Service to enable >>> Dynamic Allocation on Kuberentes. >>> >>> Anyone interested in using or working together on such a tool? >>> >>> Thanks, >>> Bo >>> >>> -- Regards, Prasad Paravatha

Re: [ANNOUNCE] Apache Spark 3.2.0

2021-10-19 Thread Prasad Paravatha
https://www.apache.org/dyn/closer.lua/spark/spark-3.2.0/spark-3.2.0-bin-hadoop3.3.tgz FYI, unable to download from this location. Also, I don’t see Hadoop 3.3 version in the dist > On Oct 19, 2021, at 9:39 AM, Bode, Meikel, NMA-CFD > wrote: > >  > Many thanks! 😊 > > From: Gengliang Wang