Re: Spark R - Loading Third Party R Library in YARN Executors

2016-08-17 Thread Shivaram Venkataraman
I think you can also pass in a zip file using the --files option (http://spark.apache.org/docs/latest/running-on-yarn.html has some examples). The files should then be present in the current working directory of the driver R process. Thanks Shivaram On Wed, Aug 17, 2016 at 4:16 AM, Felix Cheung

Re: Spark R - Loading Third Party R Library in YARN Executors

2016-08-17 Thread Felix Cheung
When you call library(), that is the library loading function in native R. As of now it does not support HDFS but there are several packages out there that might help. Another approach is to have a prefetch/installation mechanism to call HDFS command to download the R package from HDFS onto the