Hi,
why are you using add file for a json file? Cant you just read it as a
dataframe?
Regards,
Gourav Sengupta
On Fri, Aug 20, 2021 at 4:50 PM igyu wrote:
> in spark-shell
> I can run
>
> val url = "hdfs://nameservice1/user/jztwk/config.json"
> Spark.sparkContext.addFile(url)
> val json_str =
in spark-shell
I can run
val url = "hdfs://nameservice1/user/jztwk/config.json"
Spark.sparkContext.addFile(url)
val json_str = readLocalFile(SparkFiles.get(url.split("/").last))
but when I make jar package
spark-submit --master yarn --deploy-mode cluster --principal
jztwk/had...@join.com --key
