I tried running it but dint work public static final SparkConf batchConf= new SparkConf(); String master = "spark://sivarani:7077"; String spark_home ="/home/sivarani/spark-1.0.2-bin-hadoop2/"; String jar = "/home/sivarani/build/Test.jar"; public static final JavaSparkContext batchSparkContext = new JavaSparkContext(master,"SparkTest",spark_home,new String[] {jar});
public static void main(String args[]){ runSpark(0,"TestSubmit");} public static void runSpark(int crit, String dataFile){ JavaRDD<String> logData = batchSparkContext.textFile(input, 10); flatMap maptoparr reduceByKey List<Tuple2<String, Integer>> output1 = counts.collect(); } This works fine with spark-submit but when i tried to submit through code LeadBatchProcessing.runSpark(0, "TestSubmit.csv"); I get this following error HTTP Status 500 - javax.servlet.ServletException: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0.0:0 failed 4 times, most recent failure: TID 29 on host 172.18.152.36 failed for unknown reason Job aborted due to stage failure: Task 0.0:0 failed 4 times, most recent failure: TID 29 on host 172.18.152.36 failed for unknown reason Driver stacktrace: Any Advice on this? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Submiting-Spark-application-through-code-tp17452p17797.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org