Hi Adrian et al, Thanks for the advice. I haven't yet tried R.huge but will do so soon.
I have used filehash and it is working well now. A note: for some reason, R objects were not being 'flushed' from memory on R 2.5.1 - even when removed and after an explicit call to gc(). Originally I thought this was a problem with the filehash library but since found other instances when it occured and when not using filehash. After updating to R 2.6.0, this behavior resolved itself. So a note to others: if you're having problems with memory, you might make certain your version of R is up-to-date! Was this a documented issue with R 2.5.1? Best, Matt On 11/3/07, Adrian Dragulescu <[EMAIL PROTECTED]> wrote: > Matthew, > > Did you take a look at the R.huge package? It works well with numerical > data. > > If you describe the problem you're trying to solve in more detail, it may > help us give you a better solution. > > Btw, we use the filehash package with great success in accessing very large > amounts of data. > > Best, > Adrian Dragulescu > > > > On 11/1/07, Matthew Keller <[EMAIL PROTECTED]> wrote: > > > > Hi all, > > > > I've had one of my most miserable R weeks in memory. I'm trying to > > deal with huge datasets (>1GB each) but am running up against those > > pesky memory limits. The libraries filehash and g.data are not very > > suitable for what I need. I haven't gotten into the sql thing yet. > > Most recently I've been trying to install the new package ff (not yet > > on the CRAN repository). I can't find emails of the folks developing > > ff, so I was hoping someone here might have an idea how to deal with > > this. > > > > >From http://wsopuppenkiste.wiso.uni-goettingen.de/ff/ > I've downloaded > > the ff_1.0.tar.gz. > > > > > > install.packages(pkgs="ff_1.0.tar.gz",repos=NULL,destdir="/temp",type="source") > > <much output omitted> > > g++-4.0: installation problem, cannot exec > > 'i686-apple-darwin8-g++-4.0.0': No such file or directory > > make: *** [Error.o] Error 255 > > chmod: > /Library/Frameworks/R.framework/Versions/2.5/Resources/library/ff/libs/i386/*: > > No such file or directory > > ERROR: compilation failed for package 'ff' > > > > So it cannot find i686-apple-darwin8-g++-4.0.0. What the hell is that? > > A compiler I suppose. I HAVE installed the Mac OS developer tools. I > > have an MacBook Pro with 4GB RAM and am using the 32bit R: > > > > sessionInfo() > > R version 2.5.1 (2007-06-27) > > i386-apple-darwin8.9.1 > > > > locale: > > > en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 > > > > attached base packages: > > [1] "stats" "graphics" "grDevices" "utils" "datasets" > > "methods" "base" > > > > > > Any ideas how to get ff to install properly on my system? Most > > appreciative of any advice! > > > > Matt > > > > > > -- > > Matthew C Keller > > Asst. Professor of Psychology > > University of Colorado at Boulder > > www.matthewckeller.com > > > > ______________________________________________ > > R-help@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. > > > > -- Matthew C Keller Asst. Professor of Psychology University of Colorado at Boulder www.matthewckeller.com ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
