We are working on a PRs to add block partitioned matrix formats and dense
matrix multiply methods. This should be out in the next few weeks or so.
The sparse methods still need some research on partitioning schemes etc.
and we will do that after the dense methods are in place.
Thanks
Shivaram
On
ok great. when will this be ready?
On Wed, Nov 5, 2014 at 4:27 AM, Xiangrui Meng wrote:
> We are working on distributed block matrices. The main JIRA is at:
>
> https://issues.apache.org/jira/browse/SPARK-3434
>
> The goal is to support basic distributed linear algebra, (dense first
> and then
We are working on distributed block matrices. The main JIRA is at:
https://issues.apache.org/jira/browse/SPARK-3434
The goal is to support basic distributed linear algebra, (dense first
and then sparse).
-Xiangrui
On Wed, Nov 5, 2014 at 12:23 AM, ll wrote:
> @sowen.. i am looking for distribut
@sowen.. i am looking for distributed operations, especially very large
sparse matrix x sparse matrix multiplication. what is the best way to
implement this in spark?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Matrix-multiplication-in-spark-tp12562p181
Scala defines transpose.
On Thu, Aug 21, 2014 at 4:22 PM, x wrote:
> Yes.
> Now Spark API doesn't provide transpose function. You have to define it
> like below.
>
> def transpose(m: Array[Array[Double]]): Array[Array[Double]] = {
> (for {
> c <- m(0).indices
> } yield m.map(_(c))
Yes.
Now Spark API doesn't provide transpose function. You have to define it
like below.
def transpose(m: Array[Array[Double]]): Array[Array[Double]] = {
(for {
c <- m(0).indices
} yield m.map(_(c)) ).toArray
}
xj @ Tokyo
On Thu, Aug 21, 2014 at 10:12 PM, phoenix bai wrote:
> th
You could create a distributed matrix with RowMatrix.
val rmat = new RowMatrix(rows)
And then make a local DenseMatrix.
val localMat = Matrices.dense(m, n, mat)
Then multiply them.
rmat.multiply(localMat)
xj @ Tokyo
On Thu, Aug 21, 2014 at 6:37 PM, Sean Owen wrote:
> Are you trying to mul
Are you trying to multiply dense or sparse matrices? if sparse, are
they very large -- meaning, are you looking for distributed
operations?
On Thu, Aug 21, 2014 at 10:07 AM, phoenix bai wrote:
> there is RowMatrix implemented in spark.
> and I check for a while but failed to find any matrix opera
