On 29/10/2013 20:42, Rui Barradas wrote:
Hello,
You can use the argument to write.csv or write.table append = TRUE to
write the matrix in chunks. Something like the following.
That was going to be my suggestion. But the reason long vectors have not
been implemented is that is rather implausible to be useful. A text
file with the values of such a numeric matrix is likely to be 100GB.
What are you going to do with such a file? For transfer to another
program I would seriously consider a binary format (e.g. use writeBin),
as it is the conversion to and from text that is time consuming.
Some experiments suggest that it would take hours to write and at least
an hour to read such a file[*], on a very fast machine with a
start-of-the-art SSD.
[*] a file with reasonable-precision real numbers, not zeroes.
bigwrite <- function(x, file, rows = 1000L, ...){
passes <- NROW(x) %/% rows
remaining <- NROW(x) %% rows
k <- 1L
write.table(x[k:rows, ], file, row.names = FALSE, ...)
k <- k + rows
for(i in seq_len(passes)[-1]){
write.table(x[k:(rows*i), ], file, append = TRUE, row.names =
FALSE, col.names = FALSE, ...)
k <- k + rows
}
if(remaining > 0)
write.table(x[k:NROW(x), ], file, append = TRUE, row.names =
FALSE, col.names = FALSE, ...)
}
f <- "temp"
m <- matrix(0, 50012, 10)
bigwrite(m, f, sep = ",") # Use 'sep' to get a csv file
Hope this helps,
Rui Barradas
Em 29-10-2013 19:27, Petar Milin escreveu:
Hello!
I have a very large matrix of results: 50000x100000. I saved it as
RDS, but I would also need to save it as txt or csv. Is there a way to
do it? Now, with write.table I am receiving an error:
Error in .External2(C_writetable, x, file, nrow(x), p, rnames, sep,
eol, :
long vectors not supported yet: io.c:1116
Please, help! Many thanks!
PM
--
Brian D. Ripley, rip...@stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UK Fax: +44 1865 272595
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
