Re: Spark Core and ways of "talking" to it for enhancing application language support

features > supported in each language. So for Haskell, one would need to see what is > the best way to call the underlying Java API functions from Haskell and get > results back. > > Thanks > Shivaram > > On Mon, Jul 13, 2015 at 8:51 PM, Vasili I. Galchin > wrote: > >>

Spark Core and ways of "talking" to it for enhancing application language support

Hello, So far I think there are at two ways (maybe more) to interact from various programming languages with the Spark Core: PySpark API and R API. From reading code it seems that PySpark approach and R approach are very disparate ... with the latter using the R-Java bridge. Vis-a-vis/regardi

Spark application examples

Hello, Reading slides entitled "DATABRICKS" written by Holden Karau, et. al. I am also reading Spark application examples under ../spark/examples/src/main/*. Let's assume examples Driver data-manipulation.R and dataframe.R. Question: where in these Drivers are the worker "bees"

Re: language-independent RDD Spark core code?

think I found this RDD code On Fri, Jul 10, 2015 at 7:00 PM, Vasili I. Galchin wrote: > I am looking at R side, but curious what the RDD core side looks like. > Not sure which directory to look inside. ?? > > Thanks

language-independent RDD Spark core code?

I am looking at R side, but curious what the RDD core side looks like. Not sure which directory to look inside. ?? Thanks, Vasili - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h.

PySpark vs R

Hello, Just trying to get up to speed ( a week .. pls be patient with me). I have been reading several docs .. plus ... reading PySpark vs R code. I don't see an invariant between the Python and R implementations. ?? Probably I should read native Scala code, yes? Kind thx, Vasili

Re: callJMethod?

r a > socket to the JVM. This is the socket-based R to JVM bridge described in the > Spark Summit talk > https://spark-summit.org/2015/events/sparkr-the-past-the-present-and-the-future/ > > Thanks > Shivaram > > On Thu, Jul 9, 2015 at 4:28 PM, Vasili I. Galchin > wrote: &

Re: callJMethod?

nction names, arguments and sends them over a > socket to the JVM. This is the socket-based R to JVM bridge described in the > Spark Summit talk > https://spark-summit.org/2015/events/sparkr-the-past-the-present-and-the-future/ > > Thanks > Shivaram > > On Thu, Jul 9, 2015 at

callJMethod?

Hello, I am reading R code, e.g. RDD.R, DataFrame.R, etc. I see that callJMethod is repeatedly call. Is callJMethod part of the Spark Java API? Thx. Vasili

Spark and Haskell support

Hello, 1) I have been rereading kind email responses to my Spark queries. Thx. 2) I have also been reading "R" code: 1) RDD.R 2) DataFrame.R 3) All following API's => https://cwiki.apache.org/confluence/display/SPARK/Spark+Internals 4) Python ... http

Re: how can I write a language "wrapper"?

n > by Josh Rosen -- It covers some of the issues like closure capture, > serialization, JVM communication that you'll need to handle for a new > language. > > Thanks > Shivaram > > On Tue, Jun 23, 2015 at 1:35 PM, Vasili I. Galchin > wrote: >> >> Hello, &g

Re: R - Scala interface used in Spark?

Haskell?? >> >> On Fri, Jun 26, 2015 at 5:21 PM, Vasili I. Galchin >> wrote: >> >>> How about Python?? >>> >>> On Friday, June 26, 2015, Shivaram Venkataraman < >>> shiva...@eecs.berkeley.edu> wrote: >>> >>>>

Re: R - Scala interface used in Spark?

t; Thanks > Shivaram > > On Fri, Jun 26, 2015 at 3:19 PM, Vasili I. Galchin > wrote: > >> A friend sent the below: >> >> http://cran.r-project.org/web/packages/rscala/index.html >> >> Is this the "glue" between R and Scala that is used in Spark? >> >> Vasili >> > >

Re: R - Scala interface used in Spark?

mmit 2015 talk. > > Thanks > Shivaram > > On Fri, Jun 26, 2015 at 3:19 PM, Vasili I. Galchin > wrote: > >> A friend sent the below: >> >> http://cran.r-project.org/web/packages/rscala/index.html >> >> Is this the "glue" between R and Scala that is used in Spark? >> >> Vasili >> > >

R - Scala interface used in Spark?

A friend sent the below: http://cran.r-project.org/web/packages/rscala/index.html Is this the "glue" between R and Scala that is used in Spark? Vasili

Re: how can I write a language "wrapper"?

a new > language. > > Thanks > Shivaram > > On Tue, Jun 23, 2015 at 1:35 PM, Vasili I. Galchin > wrote: > >> Hello, >> >> I want to add language support for another language(other than >> Scala, Java et. al.). Where is documentation that explains to provide >> support for a new language? >> >> Thank you, >> >> Vasili >> > > >

how can I write a language "wrapper"?

Hello, I want to add language support for another language(other than Scala, Java et. al.). Where is documentation that explains to provide support for a new language? Thank you, Vasili