Just curious...
1) Are these just simple linear regression fits?
2) How many different patterns of missingness are there?
What I am thinking is that parallel processing works best on similar tasks, so
data shuffling could easily out drown out the actual regression computations.
What you might do could be to group the data according to the pattern of
missing values, and then for each group work out the regressions in matrix form
- essentially B = (X'X)^{-1}X', but could do smarter - and then do a million
regressions as BY where Y has a million columns.
-pd
> On 13 Nov 2024, at 21:02 , Ivo Welch <ivo.we...@gmail.com> wrote:
>
> I have found more general questions, but I have a specific one. I
> have a few million (independent) short regressions that I would like
> to run (each reg has about 60 observations, though they can have
> missing observations [yikes]). So, I would like to be running as many
> `lm` and `coef(lm)` in parallel as possible. my hardware is Mac, with
> nice GPUs and integrated memory --- and so far completely useless to
> me. `mclapply` is obviously very useful, but I want more, more, more
> cores.
>
> is there a recommended plug-in library to speed up just `lm` by also
> using the GPU cores?
>
> ______________________________________________
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide https://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: pd....@cbs.dk Priv: pda...@gmail.com
______________________________________________
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide https://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
