On Sat, Oct 23, 2010 at 10:07 AM, Dimitri Liakhovitski <dimitri.liakhovit...@gmail.com> wrote: > I just tried it: > > for(i in 11:16){ #i<-11 > start<-Sys.time() > print(start) > flush.console() > filename<-paste("skipped millions- ",i,".txt",sep="") > mydata<-read.csv.sql("myfilel.txt", sep="|", eol="\r\n", sql = > "select * from file limit 1000000, (1000000*i-1)")
The SQL statement does not know anything about R variables. You would need something like this: > i <- 1 > s <- sprintf("select from file limit 10, %d", 10*1-1) > s [1] "select from file limit 10, 9" > read.csv.sql(..., sql = s, ...) Also if you just want to read it in as chunks reading from a connection in R would be sufficient: k <- 5000 # no of rows per chunk first <- TRUE con <- file('myfile.csv', "r") repeat { # skip header if (first) hdgs <- readLines(con, 1) first <- FALSE x <- readLines(con, k) if (length(x) == 0) break DF <- read.csv(textConnection(x), header = FALSE) # process chunk -- we just print last row here print(tail(DF, 1)) } close(con) -- Statistics & Software Consulting GKX Group, GKX Associates Inc. tel: 1-877-GKX-GROUP email: ggrothendieck at gmail.com ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.