However, this wouldn't help much with Win XP, as this only allows for 2GB (maximum of 3 GB): http://cran.r-project.org/bin/windows/base/rw-FAQ.html#There-seems-to-be-a-limit-on-the-memory-it-uses_0021
If you want to use more RAM with windows you need to use a 64bit Version. Cheers, Henrik Am 24.07.2012 19:59, schrieb Sarah Goslee:
Sure, get more RAM. 2GB is a tiny amount if you need to load files of 1GB into R, and as you've discovered won't work. You can try a few simpler things, like making sure there's nothing loaded into R except what you absolutely need. It looks like there's no reason to read the entire file into R at once for what you want to do, so you could also load a chunk, process that, then move onto the next one. Sarah On Tue, Jul 24, 2012 at 9:45 AM, Rantony <antony.akk...@ge.com> wrote:Hi, Here in R, I need to load a huge file(.csv) , its size is 200MB. [may come more than 1GB sometimes]. When i tried to load into a variable it taking too much of time and after that when i do cbind by groups, getting an error like this " Error: cannot allocate vector of size 82.4 Mb" My requirement is, spilt data from Huge-size-file(.csv) to no. of small csv files. Here i will give no of lines to be 'split by' as input. Below i give my code ------------------------------- SplitLargeCSVToMany <- function(DataMatrix,Destination,NoOfLineToGroup) { test <- data.frame(read.csv(DataMatrix)) # create groups No.of rows group <- rep(1:NROW(test), each=NoOfLineToGroup) new.test <- cbind(test, group=group) new.test2 <- new.test new.test2[,ncol(new.test2)] <- NULL # now get indices to write out indices <- split(seq(nrow(test)), new.test[, 'group']) # now write out the files for (i in names(indices)) { write.csv(new.test2[indices[[i]],], file=paste(Destination,"data.", i, ".csv", sep=""),row.names=FALSE) } } ----------------------------------------------------- My system Configuration is, Intel Core2 Duo speed : 3GHz 2 GB RAM OS: Windows-XP [ServicePack-3] --------------------------------------------------- Any hope to solve this issue ? Thanks in advance, Antony. --
-- Dipl. Psych. Henrik Singmann PhD Student Albert-Ludwigs-Universität Freiburg, Germany http://www.psychologie.uni-freiburg.de/Members/singmann ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
