Everything is working fine now 🙂
Thanks again
Loïc
De : German Schiavon
Envoyé : mercredi 16 décembre 2020 19:23
À : Loic DESCOTTE
Cc : user@spark.apache.org
Objet : Re: Spark on Kubernetes : unable to write files to HDFS
We all been there! no reason to be
6 décembre 2020 18:01
> *À :* Loic DESCOTTE
> *Cc :* user@spark.apache.org
> *Objet :* Re: Spark on Kubernetes : unable to write files to HDFS
>
> Hi,
>
> seems that you have a typo no?
>
> Exception in thread "main" java.io.IOException: No FileSystem for scheme
Oh thank you you're right!! I feel shameful ??
De : German Schiavon
Envoyé : mercredi 16 décembre 2020 18:01
À : Loic DESCOTTE
Cc : user@spark.apache.org
Objet : Re: Spark on Kubernetes : unable to write files to HDFS
Hi,
seems that you have a ty
7;ve tried with this spark-submit option :
>
> --packages
> org.apache.hadoop:hadoop-client:2.6.5,org.apache.hadoop:hadoop-hdfs:2.6.5 \
>
> But it did't solve the issue.
> Should I add more jars?
>
> Thanks
> Loïc
> --
> *De :* Sean Ow
hdfs.impl",
classOf[org.apache.hadoop.hdfs.DistributedFileSystem].getName)
.getOrCreate()
But still the same error...
De : Sean Owen
Envoyé : mercredi 16 décembre 2020 14:27
À : Loic DESCOTTE
Objet : Re: Spark on Kubernetes : unable to write files to HDFS
