Re: [R] How to create ff objects from database connection

[Xiaobo Gu]

Hi,

         I am sorry for that I can’t determine which R-sig list to post 
questions about package ff.

 

Now I have made a little progress with this:

 

read.dbres.ffdf <- function(

         res){

         

         data1 <- fetch(res, 0)

         if (nrow(data1) == 0){

                   return (NULL)

         }

         ffd <- as.ffdf(data1)

         N <- nrow(ffd)

         

         while(!dbHasCompleted(res)){

                   data1 <- fetch(res, 0)

                   n <- nrow(data1)

                   nrow(ffd) <- N+n;

                   ffd[hi(N+1, N+n),] <- data1;

                   

                   N <- N+n  

         }

         return (ffd)

}

 

This function works well with result sets only containing integer and numeric 
columns, but has problem dealing with result sets with character , date ,time 
and timestamp columns , the error message is :

>  rs <- dbSendQuery(con, "select a, b from rtest")

> ffd <- read.dbres.ffdf(rs)

Error in ff(initdata = initdata, length = length, levels = levels, ordered = 
ordered,  : 

  unknown ffmode

 

 

Xiaobo.Gu

 

From: 顾小波 [mailto:guxiaobo1...@gmail.com] 
Sent: Sunday, August 01, 2010 10:04 PM
To: 'r-help@r-project.org'
Subject: How to create ff objects from database connection 

 

Hi

Does anybody know how to create ff objects with data reading from stream 
objects, such as data reading from PostgreSQL database through RPostgreSQL. For 
this purpose although we can save the data to a csv file through external tools 
and then read it through csv readers, but it requires one more data read and 
write operation, which is of high I/O cost for large datasets.

 

 

Xiaobo.Gu

 


        [[alternative HTML version deleted]]


______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.