Re: Missing module spark-hadoop-cloud in Maven central

2021-06-02 Thread Steve Loughran
off the record: Really irritates me too, as it forces me to do local builds even though I shouldn't have to. Sometimes I do that for other reasons, but still. Getting the cloud-storage module in was hard enough at the time that I wasn't going to push harder; I essentially stopped trying to get one

Re: [apache spark] Does Spark 2.4.8 have issues with ServletContextHandler

2021-06-02 Thread Kanchan Kauthale
Hello Sean, Please find below the stack trace- java.lang.NoclassDefFoundError: Could not initialize class org.spark.project.jetty.servlet.ServletContextHandler at org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:143) at org.apache.spark.ui.JettyUtils$.createServletHandler(J

Re: [ANNOUNCE] Apache Spark 3.1.2 released

2021-06-02 Thread Jacek Laskowski
Big shout-out to you, Dongjoon! Thank you. Pozdrawiam, Jacek Laskowski https://about.me/JacekLaskowski "The Internals Of" Online Books Follow me on https://twitter.com/jaceklaskowski On Wed, Jun 2, 2021 at 2:59 AM Dongjoon Hyu

Re: S3 Access Issues - Spark

2021-06-02 Thread Badrinath Patchikolla
Hi @KhajaAsmath Mohammed, Try these conf sets and see if you can access the s3 bucket. spark.sparkContext.hadoopConfiguration.set("fs.s3a.access.key", aws.access_id) spark.sparkContext.hadoopConfiguration.set("fs.s3a.impl", "org.apache.hadoop.fs.s3a.S3AFileSystem