I was wondering if there was any chance of getting a more distributed word2vec
implementation. I seem to be running out of memory from big local data
structures such as
val syn1Global = new Array[Float](vocabSize * vectorSize)
Is there anyway chance of getting a version where these are all pu
I wanted to ask a basic question about the types of algorithms that are
possible to apply to a DStream with Spark streaming. With Spark it is possible
to perform iterative computations on RDDs like in the gradient descent example
val points = spark.textFile(...).map(parsePoint).cache()
v
I am trying to run an example Spark standalone app with the following code
import org.apache.spark.streaming._
import org.apache.spark.streaming.StreamingContext._
object SparkGensimLDA extends App{
val ssc=new StreamingContext("local","testApp",Seconds(5))
val lines=ssc.textFileStream("/.
I am trying to run an example Spark standalone app with the following code
import org.apache.spark.streaming._
import org.apache.spark.streaming.StreamingContext._
object SparkGensimLDA extends App{
val ssc=new StreamingContext("local","testApp",Seconds(5))
val lines=ssc.textFileStream("/..
e run into this before and resolved it with a SBT clean followed by an
assembly, so maybe you could give that a try.
Let me know if that fixes it,
Andrew
2014-07-29 13:01 GMT-07:00 Alex Minnaar
mailto:aminn...@verticalscope.com>>:
I am trying to run an example Spark standalone app with th
. This error occurs when someone tries to call an abstract method.
I have run into this before and resolved it with a SBT clean followed by an
assembly, so maybe you could give that a try.
Let me know if that fixes it,
Andrew
2014-07-29 13:01 GMT-07:00 Alex Minnaar
mailto:aminn
sions of Akka. Spark depends
on a version of Akka which is different from that of Scala, and
launching a spark app using Scala command (instead of Java) can cause
issues.
TD
On Thu, Jul 31, 2014 at 6:30 AM, Alex Minnaar
wrote:
> I am eager to solve this problem. So if anyone has any suggestions
I am trying to do the same thing and also wondering what the best strategy is.
Thanks
From: ll
Sent: Wednesday, December 3, 2014 10:28 AM
To: u...@spark.incubator.apache.org
Subject: what is the best way to implement mini batches?
hi. what is the best w
I have a rowMatrix on which I want to perform two multiplications. The first
is a right multiplication with a local matrix which is fine. But after that I
also wish to right multiply the transpose of my rowMatrix with a different
local matrix. I understand that there is no functionality to tr
e without being able to transpose a rowmatrix. Am
I correct?
Thanks,
Alex
From: Reza Zadeh
Sent: Monday, January 12, 2015 1:58 PM
To: Alex Minnaar
Cc: u...@spark.incubator.apache.org
Subject: Re: RowMatrix multiplication
As you mentioned, you can perform A
?Good idea! Join each element of c with the corresponding row of A, multiply
through, then reduce. I'll give this a try.
Thanks,
Alex
From: Reza Zadeh
Sent: Monday, January 12, 2015 3:05 PM
To: Alex Minnaar
Cc: u...@spark.incubator.apache.org
Subjec
I am working with a RowMatrix and I noticed in the multiply() method that the
local matrix with which it is being multiplied is being distributed to all of
the rows of the RowMatrix. If this is the case, then is it impossible to
multiply a row matrix within a map operation? Because this would e
12 matches
Mail list logo
