On 10/08/2010 07:25 PM, Doran, Harold wrote:
These questions are OS-specific. Please provide sessionInfo() or other details
as needed
I see. I am running R on a 64 bit machine running Ubuntu 10.04
> sessionInfo()
R version 2.11.1 (2010-05-31)
x86_64-pc-linux-gnu
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=C LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
and in case it matters, this is the output of my top command
$ top
top - 19:28:21 up 8:04, 8 users, load average: 0.60, 0.72, 1.33
Tasks: 220 total, 1 running, 219 sleeping, 0 stopped, 0 zombie
Cpu(s): 10.3%us, 0.6%sy, 0.0%ni, 87.2%id, 1.9%wa, 0.0%hi, 0.0%si,
0.0%st
Mem: 6110484k total, 3847008k used, 2263476k free, 72748k buffers
Swap: 2929656k total, 0k used, 2929656k free, 2621420k cached
Cheers
Lorenzo
-----Original Message-----
From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
Sent: Friday, October 08, 2010 1:12 PM
To: r-help
Subject: [R] Memory management in R
Dear All,
I am experiencing some problems with a script of mine.
It crashes with this message
Error in grepl(fut_string, past_string) :
invalid regular expression
'12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12
Calls: entropy_estimate_hash -> total_entropy_lz -> entropy_lz -> grepl
In addition: Warning message:
In grepl(fut_string, past_string) : regcomp error: 'Out of memory'
Execution halted
To make a long story short, I use some functions which eventually call
grepl on very long strings to check whether a certain substring is part
of a longer string.
Now, the script technically works (it never crashes when I run it on a
smaller dataset) and the problem does not seem to be RAM memory (I have
several GB of RAM on my machine and its consumption never shoots up so
my machine never resorts to swap memory).
So (though I am not an expert) it looks like the problem is some
limitation of grepl or R memory management.
Any idea about how I could tackle this problem or how I can profile my
code to fix it (though it really seems to me that I have to find a way
to allow R to process longer strings).
Any suggestion is appreciated.
Cheers
Lorenzo
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
