Hi all, I am trying to read a large csv file (~11 Gb - ~900,000 columns, 3000 rows) using the read.big.matrix command from the bigmemory package. I am using the following command:
x<-read.big.matrix('data.csv', sep=',', header=TRUE, type='char', backingfile='data.bin', descriptorfile='data.desc') When the command starts, everything seems to be fine, and when examining the CPU being used on my linux system, R is using 100%. Within about 10 minutes, the data.bin file is created, but then the CPU use drops to 0%, where it remains for at least 24 hours (I have let the program continue for up to 48 hours, but after that I kill the command or R session). The data.desc file never seems to be created. Are there any estimates on how long this process should take? Unfortunately, I am not getting any error messages, so I am not sure if the command is continuing to work (despite using 0% CPU) or if it has crashed or if I am just being too impatient! I am running version 2.10.0 of R with version of 3.12 of bigmemory on a Linux server with 8 cores and 24 GB of RAM. Any help/advice would be greatly appreciated! Thanks, Eric Claus ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.