Hi Pol Thanks for the reply!
"spark:latest" is based on the Dockerfile included with Spark Release 3.1.1 - "Pre-build for Apache Hadoop 3.2 and later" I have simplified my Dockerfile quite a bit, but its still not loading at all. FROM spark:latest ARG jetty_version=9.4.40.v20210413 ARG wildfly_version=1.0.7.Final ARG azure_storage_version=7.0.0 ARG azure_keyvault_version=1.2.4 ARG azure_hadoop=3.2.2 USER root ADD https://search.maven.org/remotecontent?filepath=org/apache/hadoop/hadoop-azure/${azure_hadoop}/hadoop-azure-${azure_hadoop}.jar $SPARK_HOME/jars/hadoop-azure-${azure_hadoop}.jar ADD https://search.maven.org/remotecontent?filepath=com/microsoft/azure/azure-keyvault-core/${azure_keyvault_version}/azure-keyvault-core-${azure_keyvault_version}.jar $SPARK_HOME/jars/azure-keyvault-core-${azure_keyvault_version}.jar ADD https://search.maven.org/remotecontent?filepath=com/microsoft/azure/azure-storage/${azure_storage_version}/azure-storage-${azure_storage_version}.jar $SPARK_HOME/jars/azure-storage-${azure_storage_version}.jar ADD https://search.maven.org/remotecontent?filepath=org/eclipse/jetty/jetty-util/${jetty_version}/jetty-util-${jetty_version}.jar $SPARK_HOME/jars/jetty-util-${jetty_version}.jar ADD https://search.maven.org/remotecontent?filepath=org/eclipse/jetty/jetty-util-ajax/${jetty_version}/jetty-util-ajax-${jetty_version}.jar $SPARK_HOME/jars/jetty-util-ajax-${jetty_version}.jar ADD https://search.maven.org/remotecontent?filepath=org/wildfly/openssl/wildfly-openssl/${wildfly_version}/wildfly-openssl-${wildfly_version}.jar $SPARK_HOME/jars/wildfly-openssl-${wildfly_version}.jar RUN chmod -R 775 $SPARK_HOME/jars USER 185 If you have any suggestions on finding the "correct" hadoop-azure version and its dependencies I am all ears. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
