Le mercredi 06 mars 2013 à 09:18 -0800, Jonas125 a écrit :
> Length(Datasplit) = 7100
>
> I did a regression for Datasplit[[1]] and the calculated columns --> the
> object size is 70 MB. Quite large....
7100*70/1024 = 485 (GB)
No wonder why you run out of memory quite fast.
You probably do not need to store the whole lm objects: usually you need
coefficients, R-squared, things like that. So instead of returning the
objects, return a vector or a list with only the elements you need, you
will save much space.
And if you really need the objects, set these lm() arguments to FALSE to
make the result smaller:
model, x, y, qr: logicals. If ‘TRUE’ the corresponding components of
the fit (the model frame, the model matrix, the response, the
QR decomposition) are returned.
> Assuming that R cannot handle inf values in regressions (didn't have the
> time to google it)
> How can I avoid the calculation of infinite values? Like "If the denominator
> would be zero, choose 0.0000001 as the denominator instead."
> Dataset[is.infinite(Dataset)] <- 0 does not work for me --> "default method
> not implemented for type 'list' "
> class(Dataset) = data.frame
I don't understand why you think infinite values can trigger a memory
problem. Why don't you just try it?
> lm(c(1, Inf) ~ c(1, 2))
Erreur dans lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) :
NA/NaN/Inf in 'y'
> lm(c(1, 2) ~ c(1, Inf))
Erreur dans lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) :
NA/NaN/Inf in 'x'
So, if anything, this would stop your lapply() call sooner or later, and
save your machine from freezing.
Regards
>
>
> --
> View this message in context:
> http://r.789695.n4.nabble.com/lm-Regression-takes-24-GB-RAM-Error-message-tp4660434p4660501.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
