Re: [R] Exporting large datasets

2009-07-01 Thread DrDimo
Thanks guys, writing smaller pieces did the trick. Still takes about 4 hours to do but that more manageable than over 30 hours without even writing a line. jholtman wrote: > > What do you want to export it to and in what format? You will probably > have > a problem trying to write that large a

Re: [R] Exporting large datasets

2009-06-29 Thread Allan Engelhardt
Depends on what you mean by "export". I like save(). Check out the options in the help. It seems to be very efficient and I had no problems working with the 50,000 x 15,000 data sets from the 2009 KDD Cup challenge. Maybe your disk is just slow (network file storage)? Allan. On 29/06/09 07

Re: [R] Exporting large datasets

2009-06-29 Thread jim holtman
What do you want to export it to and in what format? You will probably have a problem trying to write that large an object out at one time. Try instead opening a connecting and the writing much smaller pieces to the file: x <- matrix(runif(1), 100) x[,1] <- 1:100 # put some labels for testi

[R] Exporting large datasets

2009-06-29 Thread DrDimo
Hi all, I am currently working on a particularly large (data wise) project, the problem I am having is with exporting the data (for use later on). My current sample dataset contains about 400million entries (approximately 2 columns x 2 rows). I tried using the write.table command, but 8