Hi all,
I am trying to read a large csv file (~11 Gb - ~900,000 columns, 3000
rows) using the read.big.matrix command from the bigmemory package.  I
am using the following command:

x<-read.big.matrix('data.csv', sep=',', header=TRUE, type='char',
backingfile='data.bin', descriptorfile='data.desc')

When the command starts, everything seems to be fine, and when
examining the CPU being used on my linux system, R is using 100%.
Within about 10 minutes, the data.bin file is created, but then the
CPU use drops to 0%, where it remains for at least 24 hours (I have
let the program continue for up to 48 hours, but after that I kill the
command or R session).  The data.desc file never seems to be created.
Are there any estimates on how long this process should take?
Unfortunately, I am not getting any error messages, so I am not sure
if the command is continuing to work (despite using 0% CPU) or if it
has crashed or if I am just being too impatient!

I am running version 2.10.0 of R with version of 3.12 of bigmemory on
a Linux server with 8 cores and 24 GB of RAM.

Any help/advice would be greatly appreciated!

Thanks,
Eric Claus

______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Reply via email to