Hi
I am trying to write a simple program using addFile Function but getting
error in my worker node that file doest not exist
tage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost
task 0.3 in stage 0.0 (TID 3, slave2.novalocal):
java.io.FileNotFoundException: File
file:/tmp/spark-2791415c-8c20-4920-b3cd-5a6b8b6f3f8d/userFiles-a5a98f06-2d38-48
76-8c8c-82a10ac5431f/csv_ip.csv does not exist
code are as below :
val sc = new SparkContext(sparkConf)
val inputFile
="file:///home/ubuntu/test/Spark_CSV/spark_csv_job/csv_ip.csv"
sc.addFile(inputFile)
val inFile = sc.textFile("file://"+SparkFiles.get("csv_ip.csv"))
inFile.take(10).foreach(println)
please help me resolve error. Thanks in advance.
Regards
prateek
