Re: Matrix Multiplication of two RDD[Array[Double]]'s

2014-05-18 Thread Andrew Ash
Hi Liquan, There is some working being done on implementing linear algebra algorithms on Spark for use in higher-level machine learning algorithms. That work is happening in the MLlib project, which has a org.apache.spark.mllib.linalgpackage you may find useful. See https://github.com/apache/spa

Matrix Multiplication of two RDD[Array[Double]]'s

2014-05-17 Thread Liquan Pei
Hi I am currently implementing an algorithm involving matrix multiplication. Basically, I have matrices represented as RDD[Array[Double]]. For example, If I have A:RDD[Array[Double]] and B:RDD[Array[Double]] and what would be the most efficient way to get C = A * B Both A and B are large, so it w