yes, the context is being closed at the end. ________________________________ From: Akhil Das <ak...@sigmoidanalytics.com> Sent: Sunday, August 30, 2015 9:03 AM To: Pranay Tonpay Cc: user@spark.apache.org Subject: Re: spark-submit issue
Did you try putting a sc.stop at the end of your pipeline? Thanks Best Regards On Thu, Aug 27, 2015 at 6:41 PM, pranay <pranay.ton...@impetus.co.in<mailto:pranay.ton...@impetus.co.in>> wrote: I have a java program that does this - (using Spark 1.3.1 ) Create a command string that uses "spark-submit" in it ( with my Class file etc ), and i store this string in a temp file somewhere as a shell script Using Runtime.exec, i execute this script and wait for its completion, using process.waitFor Doing ps -ef shows me SparkSubmitDriverBootstrapper , the script running my class ... parent child relationship.. The job gets triggered on spark-cluster and gets over but SparkSubmitDriverBootstrapper still shows up, due to this the process.waitFor never comes out and i can't detect the execution end... If i run the /temp file independently. things work file... only when i trigger /temp scrict inside Runtime.exec , this issue occurs... Any comments ? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-submit-issue-tp24474.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org<mailto:user-unsubscr...@spark.apache.org> For additional commands, e-mail: user-h...@spark.apache.org<mailto:user-h...@spark.apache.org> ________________________________ NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
