On Wed, 28 Oct 2009, Kingsford Jones wrote:

I thought I'd share this with the list since it appears to provide a
quick fix to some memory problems, and I haven't see it discussed in
relation to R.

Seems you didn't look in the places mentioned in the posting guide, fo this is an actual FAQ!

See the references in rw-FAQ Q2.9 (and this has been documented there for many years, and the R developer made R large-address-aware so users who looked at the rw-FAQ could benefit from this).

See http://msdn.microsoft.com/en-us/library/bb613473%28VS.85%29.aspx for a more rounded view covering all the relevant OSes.

To reallocate virtual memory from kernel-mode to user-mode in 32-bit
Vista or Windows 7 one can use the increaseuserva boot option value.
See
http://msdn.microsoft.com/en-us/library/aa906211.aspx

On my 4GB Vista machine, R is now able to use 3GB memory (previously
at the default value of 2GB).

Here's the method:

Open cmd.exe as administrator and type

BCDEdit /set IncreaseUserVA <value>

where <value> is between 2048 and 3072 (ie 2-3GB), then reboot.  Given
that you've set the --max-mem-size flag, or used the memory.limit
function in an R session to increase R's memory allocation, your R
processes should now be allowed to access up to 3GB of virtual memory.

See the rw-FAQ: it is 3GB of address space and 2.5GB of memory by default.

I am not a Windows expert, so if anyone knows of disadvantages of
using this method *please* post a response.

This is not advised in general by those who are experts (why do you think that it is not the default in Windows?). As the URL above says

  'For applications that are memory-intensive, such as database
  management systems (DBMS), the use of a larger virtual address space
  can provide considerable performance and scalability benefits.
  However, the file cache, paged pool, and nonpaged pool are smaller,
  which can adversely affect applications with heavy networking or
  I/O.  Therefore, you might want to test your application under load,
  and examine the performance counters to determine whether your
  application benefits from the larger address space.'

Remember that you are setting this for all uses and users of your machine, not just the R process (so the last sentence is somewhat misleading).

The best advice seems to be that if you have 4GB or more of RAM is to run a 64-bit version of Windows, which gives a 32-bit process a 4GB user address space. If you have a lot more memory, consider a 64-bit build of R (and an OS that handles memory more efficiently: one of my background tasks right now is to investigate why R under Windows is running a particular memory-intensive multiprocessor task so much more slowly than Linux on the same hardware).

best,

Kingsford Jones

--
Brian D. Ripley,                  rip...@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Reply via email to