Driver crash at the end with InvocationTargetException when running SparkPi

al, and my error is caused by Hadoop.util.Shell.runCommand. Best Regards Dong Lei

RE: Driver crash at the end with InvocationTargetException when running SparkPi

Thanks Akhil so such! It turns out to be HADOOP_HOME not set. Dong Lei From: Akhil Das [mailto:ak...@sigmoidanalytics.com] Sent: Monday, June 8, 2015 3:12 PM To: Dong Lei Cc: user@spark.apache.org Subject: Re: Driver crash at the end with InvocationTargetException when running SparkPi Can you

ClassNotDefException when using spark-submit with multiple jars and files located on HDFS

will spark copy these files into one folder? BTW, my class works in a client mode with all jars and files in local. Thanks Dong Lei

RE: ClassNotDefException when using spark-submit with multiple jars and files located on HDFS

application jar but not the other jars specified by “—jars”. Or I misunderstand the usage of “--jars”, and the jars should be already in every worker, driver will not download them? Is there some useful docs? Thanks Dong Lei From: Akhil Das [mailto:ak...@sigmoidanalytics.com] Sent: Tuesday, June 9, 2015

RE: ClassNotDefException when using spark-submit with multiple jars and files located on HDFS

jars and not properly downloaded from HDFS to these folders? I’m using standalone mode. Any ideas？ Thanks Dong Lei From: Akhil Das [mailto:ak...@sigmoidanalytics.com] Sent: Tuesday, June 9, 2015 4:46 PM To: Dong Lei Cc: user@spark.apache.org Subject: Re: ClassNotDefException when using spark

RE: ClassNotDefException when using spark-submit with multiple jars and files located on HDFS

can I ask to support such a scenario? Should I send to dev@spark? Does yarn-cluster mode support dependency jars and files on HDFS? Thanks Dong Lei From: Jörn Franke [mailto:jornfra...@gmail.com] Sent: Wednesday, June 10, 2015 12:48 PM To: Dong Lei; Akhil Das Cc: user@spark.apache.org Subject: Re