Re: Error when using multiple python files spark-submit

2015-03-20 Thread Guillaume Charhon
I see. I will try the other way around. On Thu, Mar 19, 2015 at 8:06 PM, Davies Liu wrote: > the options of spark-submit should come before main.py, or they will > become the options of main.py, so it should be: > > ../hadoop/spark-install/bin/spark-submit --py-files > > > /home/poiuytrez

Re: Spark-submit and multiple files

2015-03-20 Thread Guillaume Charhon
Hi Davies, I am already using --py-files. The system does use the other file. The error I am getting is not trivial. Please check the error log. On Thu, Mar 19, 2015 at 8:03 PM, Davies Liu wrote: > You could submit additional Python source via --py-files , for example: > > $ bin/spark-submit

Re: Movie Recommendation tutorial

2015-02-24 Thread Guillaume Charhon
I am using Spark 1.2.1. Thank you Krishna, I am getting almost the same results as you so it must be an error in the tutorial. Xiangrui, I made some additional tests with lambda to 0.1 and I am getting a much better rmse: RMSE (validation) = 0.868981 for the model trained with rank = 8, lambda =