Dear Berend,
thanks to everyone who responded. I should point that the data matrix is not
sparse. Yes, the irlba package seemed of interest, however annoyingly, they
don't specify in the package how large is "large"!!!, and when you read the
paper on which it is based, it deals with rather smaller data matrices than the
ones I have, so it is unclear how it would perform and scale.
So, no one has direct experience running irlba on a data matrix as large as
500,000 x 1,000 or larger?
kind regards
Andy
________________________________
From: Berend Hasselman <b...@xs4all.nl>
Cc: "r-help@R-project.org" <r-help@r-project.org>
Sent: Monday, 8 April 2013, 20:31
Subject: Re: [R] SVD on very large data matrix
>
>
> Dear All,
>
> I need to perform a SVD on a very large data matrix, of dimension ~ 500,000 x
> 1,000 , and I am looking
> for an efficient algorithm that can perform an approximate (partial) SVD to
> extract on the order of the top 50
> right and left singular vectors.
>
> Would be very grateful for any advice on what R-packages are available to
> perform such a task, what the RAM requirement is, and indeed what would be
> the state-of-the-art in terms of numerical algorithms and programming
> language to use to accomplish this task.
Info found with package sos and findFn("svd") and scrolling through the list
for something relevant.
Have a look at package irlba.
It can work with dense matrices and sparse matrices as provided by package
Matrix, according to the documentation.
Berend
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
