Re: Submit custom python packages from current project

2016-02-19 Thread Eike von Seggern
Hello, 2016-02-16 11:03 GMT+01:00 Mohannad Ali : > Hello Everyone, > > I have code inside my project organized in packages and modules, however I > keep getting the error "ImportError: No module named " when > I run spark on YARN. > > My directory structure is something like this: > > project/ >

Re: Submit custom python packages from current project

2016-02-16 Thread Mohannad Ali
Hello Ramanathan, Unfortunately I tried this already and it doesn't work. Mo On Tue, Feb 16, 2016 at 2:13 PM, Ramanathan R wrote: > Have you tried setting PYTHONPATH? > $ export PYTHONPATH="/path/to/project" > $ spark-submit --master yarn-client /path/to/project/main_script.py > > Regards, > R

Re: Submit custom python packages from current project

2016-02-16 Thread Ramanathan R
Have you tried setting PYTHONPATH? $ export PYTHONPATH="/path/to/project" $ spark-submit --master yarn-client /path/to/project/main_script.py Regards, Ram On 16 February 2016 at 15:33, Mohannad Ali wrote: > Hello Everyone, > > I have code inside my project organized in packages and modules, ho