Re: Aggregating over sorted data

This is also a feature we need for our time-series processing > On 19 Dec 2016, at 04:07, Liang-Chi Hsieh wrote: > > > Hi, > > As I know, Spark SQL doesn't provide native support for this feature now. > After searching, I found only few database systems support it, e.g., > PostgreSQL. > >

Re: [GRAPHX] Graph Algorithms and Spark

-synchronous parallel processing that is the foundation of most of the above algorithms. We cover other algorithms in our book and if you search on google you will find a number of other examples. --- Robin East Spark GraphX

Re: spark graphx storage RDD memory leak

this looks like https://issues.apache.org/jira/browse/SPARK-12655 <https://issues.apache.org/jira/browse/SPARK-12655> fixed in 2.0 --- Robin East Spark GraphX in Action Michael Malak and Robin East Manning Publicati

Re: Shared memory between C++ process and Spark

not the case. If you didn’t mean then we are both in agreement. --- Robin East Spark GraphX in Action Michael Malak and Robin East Manning Publications Co. http://www.manning.com/books/spark-graphx-in-action <h

Re: Shared memory between C++ process and Spark

.. >>> Hi, Robin, >>> Thanks for your reply and thanks for copying my question to user mailing >>> list. >>> Yes, we have a distributed C++ application, that will store data on each >>> node in the cluster, and we hope to leverage Spark to do more fancy

Re: Shared memory between C++ process and Spark

ch of the current functionality they support... >>> Hi, Robin, >>> Thanks for your reply and thanks for copying my question to user mailing >>> list. >>> Yes, we have a distributed C++ application, that will store data on each >>> node in the

Re: Shared memory between C++ process and Spark

architectural sense. --- Robin East Spark GraphX in Action Michael Malak and Robin East Manning Publications Co. http://www.manning.com/books/spark-graphx-in-action <http://www.manning.com/books/spark-graphx-in-act

Re: query on SVD++

ect%20=%20SPARK%20AND%20resolution%20=%20Unresolved%20AND%20component%20=%20GraphX%20ORDER%20BY%20updated%20DESC> for the latest. --- Robin East Spark GraphX in Action Michael Malak and Robin East Manning Publicati

Re: Add a function to support Google's Word2Vec

Have a look at SPARK-9484, JIRA is already there. Pull request would be good. Robin > On 17 Nov 2015, at 12:10, yuming wang wrote: > > Hi: > > > > I have a function to load Google’s Word2Vec generated binary file and spark > can use this model. If it is convenient, I'm going to open a JIR

Re: [VOTE] Release Apache Spark 1.4.1 (RC2)

I used the following build command: build/mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package this also gave the ‘Dependency-reduced POM’ loop Robin > On 3 Jul 2015, at 23:41, Patrick Wendell wrote: > > What if you use the built-in maven (i.e. build/mvn). It might be that

Re: Can not build master

Yes me too > On 3 Jul 2015, at 22:21, Ted Yu wrote: > > This is what I got (the last line was repeated non-stop): > > [INFO] Replacing original artifact with shaded artifact. > [INFO] Replacing > /home/hbase/spark/bagel/target/spark-bagel_2.10-1.5.0-SNAPSHOT.jar with > /home/hbase/spark/bagel/

Re: LDA and PageRank Using GraphX

There is an LDA example in the MLlib examples. You can run it like this: ./bin/run-example mllib.LDAExample --stopwordFile stop words is a file of stop words, 1 on each line. Input documents are the text of each document, 1 document per line. To see all the options just run with no options or

Re: [VOTE] Release Apache Spark 1.3.0 (RC2)

+1 (subject to comments on ec2 issues below) machine 1: Macbook Air, OSX 10.10.2 (Yosemite), Java 8 machine 2: iMac, OSX 10.8.4, Java 7 1. mvn clean package -DskipTests (33min/13min) 2. ran SVM benchmark https://github.com/insidedctm/spark-mllib-benchmark EC2 issues: 1) Unable to successfully

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

Running ec2 launch scripts gives me the following error: ssl.SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed Full stack trace at https://gist.github.com/insidedctm/4d41600bc22560540a26 I’m running OSX Mavericks 10.9.5 I’ll inves

Fwd: LinearRegressionWithSGD accuracy

Sent from my iPhone Begin forwarded message: > From: Robin East > Date: 16 January 2015 11:35:23 GMT > To: Joseph Bradley > Cc: Yana Kadiyska , Devl Devel > > Subject: Re: LinearRegressionWithSGD accuracy > > Yes with scaled data intercept would be 5000 but

Re: LinearRegressionWithSGD accuracy

-dev, +user You’ll need to set the gradient descent step size to something small - a bit of trial and error shows that 0.0001 works. You’ll need to create a LinearRegressionWithSGD instance and set the step size explicitly: val lr = new LinearRegressionWithSGD() lr.optimizer.setStepSize(0.