RE: SparkR package path

2015-09-24 Thread Sun, Rui
Subject: Re: SparkR package path Right now in sparkR.R the backend hostname is hard coded to "localhost" (https://github.com/apache/spark/blob/master/R/pkg/R/sparkR.R#L156). If we make that address configurable / parameterized, then a user can connect a remote Spark cluster with no ne

RE: SparkR package path

2015-09-24 Thread Sun, Rui
AM To: Sun, Rui Cc: shiva...@eecs.berkeley.edu; dev@spark.apache.org Subject: Re: SparkR package path Requiring users to download entire Spark distribution to connect to a remote cluster (which is already running Spark) seems an over kill. Even for most spark users who download Spark source, it

Re: SparkR package path

2015-09-24 Thread Luciano Resende
or normal users >> who >> >> are not developers, they are not required to download Spark source, >> build >> >> and install SparkR package. They just need to download a Spark >> distribution, >> >> and then use SparkR. >> >> >> &

Re: SparkR package path

2015-09-24 Thread Hossein
, > >> and then use SparkR. > >> > >> > >> > >> For using SparkR in Rstudio, there is a documentation at > >> https://github.com/apache/spark/tree/master/R > >> > >> > >> > >> > >> > >> > >

Re: SparkR package path

2015-09-24 Thread Shivaram Venkataraman
rkR package. They just need to download a Spark distribution, >> and then use SparkR. >> >> >> >> For using SparkR in Rstudio, there is a documentation at >> https://github.com/apache/spark/tree/master/R >> >> >> >> >> >> >>

Re: SparkR package path

2015-09-24 Thread Hossein
er 24, 2015 1:42 AM > *To:* shiva...@eecs.berkeley.edu > *Cc:* Sun, Rui; dev@spark.apache.org > *Subject:* Re: SparkR package path > > > > Yes, I think exposing SparkR in CRAN can significantly expand the reach of > both SparkR and Spark itself to a larger community of data scientist

RE: SparkR package path

2015-09-23 Thread Sun, Rui
documentation at https://github.com/apache/spark/tree/master/R From: Hossein [mailto:fal...@gmail.com] Sent: Thursday, September 24, 2015 1:42 AM To: shiva...@eecs.berkeley.edu Cc: Sun, Rui; dev@spark.apache.org Subject: Re: SparkR package path Yes, I think exposing SparkR in CRAN can significantly

Re: SparkR package path

2015-09-23 Thread Hossein
Yes, I think exposing SparkR in CRAN can significantly expand the reach of both SparkR and Spark itself to a larger community of data scientists (and statisticians). I have been getting questions on how to use SparkR in RStudio. Most of these folks have a Spark Cluster and wish to talk to it from

Re: SparkR package path

2015-09-22 Thread Shivaram Venkataraman
As Rui says it would be good to understand the use case we want to support (supporting CRAN installs could be one for example). I don't think it should be very hard to do as the RBackend itself doesn't use the R source files. The RRDD does use it and the value comes from https://github.com/apache/s

RE: SparkR package path

2015-09-21 Thread Sun, Rui
Hossein, Any strong reason to download and install SparkR source package separately from the Spark distribution? An R user can simply download the spark distribution, which contains SparkR source and binary package, and directly use sparkR. No need to install SparkR package at all. From: Hosse