Re: Spark 3 pod template for the driver

Hello, Adding the dev mailing list maybe there is someone here that can help to have/show a valid/accepted pod template for spark 3? Thanks in advance, Michel Le ven. 26 juin 2020 à 14:03, Michel Sumbul a écrit : > Hi Jorge, > If I set that in the spark submit command it works but I w

Re: Spark 3 pod template for the driver

Hi Edeesis, The goal is to not have these settings in the spark submit command. If I specify the same things in a pod template for the executor, I still got the message: "Exception in thread "main" org.apache.spark.SparkException "Must specify the driver container image" it even don't try to star

Spark3 on k8S reading encrypted data from HDFS with KMS in HA

Hi guys, Does anyone try Spark3 on k8s reading data from HDFS encrypted with KMS in HA mode (with kerberos)? I have a wordcount job running with Spark3 reading data on HDFS (hadoop 3.1) everything secure with kerberos. Everything works fine if the data folder is not encrypted (spark on k8s). If t

Re: Spark3 on k8S reading encrypted data from HDFS with KMS in HA

d with KMS :-) Thanks, Michel Le jeu. 13 août 2020 à 14:32, Michel Sumbul a écrit : > Hi guys, > > Does anyone try Spark3 on k8s reading data from HDFS encrypted with KMS in > HA mode (with kerberos)? > > I have a wordcount job running with Spark3 reading data on HDFS (hadoop &

Re: [UPDATE] Apache Spark 3.1.0 Release Window

I think you put Jan 2020 instead of 2021 :-) Le mar. 13 oct. 2020 à 00:51, Xiao Li a écrit : > Thank you, Dongjoon > > Xiao > > On Mon, Oct 12, 2020 at 4:19 PM Dongjoon Hyun > wrote: > >> Hi, All. >> >> Apache Spark 3.1.0 Release Window is adjusted like the following today. >> Please check the