It maybe the Spark executor is running as a different user and it can't see where RScript is?
You might want to try putting Rscript path to PATH. Also please see this for the config property to set for the R command to use: https://spark.apache.org/docs/latest/configuration.html#sparkr _____________________________ From: ian.malo...@tdameritrade.com<mailto:ian.malo...@tdameritrade.com> Sent: Wednesday, November 9, 2016 12:12 PM Subject: Issue Running sparkR on YARN To: <user@spark.apache.org<mailto:user@spark.apache.org>> Hi, I'm trying to run sparkR (1.5.2) on YARN and I get: java.io.IOException: Cannot run program "Rscript": error=2, No such file or directory This strikes me as odd, because I can go to each node and various users and type Rscript and it works. I've done this on each node and spark-env.sh as well: export R_HOME=/path/to/R This is how I'm setting it on the nodes (/etc/profile.d/path_edit.sh): export R_HOME=/app/hdp_app/anaconda/bin/R PATH=$PATH:/app/hdp_app/anaconda/bin Any ideas? Thanks, Ian --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org<mailto:user-unsubscr...@spark.apache.org>
