Re: [Rd] unexpectedly high memory use in R 2.14.0

2012-04-12 Thread andre zege
Henrik, thanks for your reply. I might have misrepresented a bit my actual code . It seems that you are suggesting doing rm() on objects i don't use. In the real code which behavior i reported it is exactly what is being done, i.e i use rm(). I also use a small wrapper around load that lets me as

Re: [Rd] unexpectedly high memory use in R 2.14.0

2012-04-11 Thread Henrik Bengtsson
Leaving aside what's going on inside abind::abind(), maybe the following sheds some light on what's is being wasted: # Preallocate (probably doesn't make a difference because it's a list) mat.data <- vector("list", length=length(files)); for (j in 1:length(files)){ vars <- load(file.path(dump

Re: [Rd] unexpectedly high memory use in R 2.14.0

2012-04-11 Thread andre zege
You are quite right that my exec time would seriously go down if i pre-allocate and not even use abind, just assign into the preallocated matrix. The reason i didn't do it here is that this is a part of some utility function that doesn't know the sizes of chunks that are on disk untill it reads all

Re: [Rd] unexpectedly high memory use in R 2.14.0

2012-04-11 Thread peter dalgaard
On Apr 12, 2012, at 00:53 , andre zege wrote: > I recently started using R 2.14.0 on a new machine and i am experiencing > what seems like unusually greedy memory use. It happens all the time, but > to give a specific example, let's say i run the following code > > > > for(j in 1:leng

[Rd] unexpectedly high memory use in R 2.14.0

2012-04-11 Thread andre zege
I recently started using R 2.14.0 on a new machine and i am experiencing what seems like unusually greedy memory use. It happens all the time, but to give a specific example, let's say i run the following code for(j in 1:length(files)){ load(file.path(dump.dir, files[j])) ma