Which py file is your main file (primary py file)? Zip the other two py files. Leave the main py file alone. Don't copy them to S3 because it seems that only local primary and additional py files are supported.
./bin/spark-submit --master spark://... --py-files <zip file> <main py file> -----Original Message----- From: kevllino [mailto:kevin.e...@mail.dcu.ie] Sent: Tuesday, April 12, 2016 5:07 PM To: user@spark.apache.org Subject: Run a self-contained Spark app on a Spark standalone cluster Hi, I need to know how to run a self-contained Spark app (3 python files) in a Spark standalone cluster. Can I move the .py files to the cluster, or should I store them locally, on HDFS or S3? I tried the following locally and on S3 with a zip of my .py files as suggested here <http://spark.apache.org/docs/latest/submitting-applications.html> : ./bin/spark-submit --master spark://ec2-54-51-23-172.eu-west-1.compute.amazonaws.com:5080 --py-files s3n://AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY@mubucket//weather_predict.zip But get: “Error: Must specify a primary resource (JAR or Python file)” Best, Kevin -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Run-a-self-contained-Spark-app-on-a-Spark-standalone-cluster-tp26753.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
