Not sure about the performance, but for now you could do:
val mat = new IndexedRowMatrix(...)
.toCoordinateMatrix()
.transpose()
.toRowMatrix()
On Mon, Aug 31, 2015 at 1:31 PM, Reza Zadeh wrote:
> This is ongoing w
15 you've mentioned)
> e.g. if you have 15 * 8 cores but your rdd with 1000 partitions - there is
> no way you'll get parallel job execution since only 1 job already fills all
> cores with tasks(unless you are going to manage resources per each
> submit/job)
>
>
>
&g
Hi, I have a large number of RDDs that I need to process separately.
Instead of submitting these jobs to the Spark scheduler one by one, I'd
like to submit them in parallel in order to maximize cluster utilization.
I've tried to process the RDDs as Futures, but the number of Active jobs
maxes out
