Re: Submitting multiple files pyspark

Hi Cheng, You specify extra python files through --py-files. For example: bin/spark-submit [your other options] --py-files helper.py main_app.py -Andrew 2014-08-27 22:58 GMT-07:00 Chengi Liu : > Hi, > I have two files.. > > main_app.py and helper.py > main_app.py calls some functions in hel

Submitting multiple files pyspark

Hi, I have two files.. main_app.py and helper.py main_app.py calls some functions in helper.py. I want to use spark-submit to submit a job but how do i specify helper.py? Basically, how do i specify multiple files in spark? Thanks