I am trying to write a simple program using addFile Function but getting
error in my worker node that file doest not exist
tage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost
task 0.3 in stage 0.0 (TID 3, slave2.novalocal):
java.io.FileNotFoundException: File
file:/tmp/spark-2791415c-8c20-4920-b3cd-5a6b8b6f3f8d/userFiles-a5a98f06-2d38-48
76-8c8c-82a10ac5431f/csv_ip.csv does not exist
code are as below :
val sc = new SparkContext(sparkConf)
val inputFile ="file:///home/ubuntu/test/Spark_CSV/spark_csv_job/csv_ip.csv"
sc.addFile(inputFile)
val inFile = sc.textFile("file://"+SparkFiles.get("csv_ip.csv"))
inFile.take(10).foreach(println)
please help me resolve error. Thanks in advance.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/get-java-io-FileNotFoundException-when-use-addFile-Function-tp23867.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
