I'm trying to use sqdf's function read.csv.sql to read CSV files in which the
missing values are represented by NA's.
Plain old read.csv works fine on these files, but they are rather large and I'd
like to filter using sql-like statements.
However, even if I specify field.types correctly and nrows=-1, it still turns
the columns with NA's into chars or 0.
I'm trying to make this OS independent, so I don't think I can use a filter to
convert the NA's to NULL's or whatever SQLite would understand.
I can accept it everything has to be read in as char and then convert to
doubles with as.numeric, but I'm looking for speed.
Here is code I thought would read the file (I've attached a small sample.)
It almost works if there are no NA's in the initial rows, but it still turns
NA's into 0's instead of NA or something I can change into NA;
and it returns characters if there are NA's in the initial rows.
(0 is a possible value so I can't filter out the 0's.)
field.types <- list(V1='char', V2='char', V3='real', V4='int', V5='real',
V6='int', V7='real')
dtst <- read.csv.sql("./tmp.csv", header=FALSE, field.types=field.types,
nrows=-1)
str(dtst)
'data.frame': 32 obs. of 7 variables:
$ V1: chr "2012-07-01" "2012-07-01" "2012-07-01" "2012-07-01" ...
$ V2: chr "15:50:00" "15:51:00" "15:52:00" "15:53:00" ...
$ V3: chr "NA" "NA" "NA" "NA" ...
$ V4: int 0 0 0 0 0 0 0 0 0 0 ...
$ V5: chr "NA" "NA" "NA" "NA" ...
$ V6: int 0 0 0 0 0 0 0 0 0 0 ...
$ V7: chr "NA" "NA" "NA" "NA" ...
Any suggestions?
Thanks,
-- David L. Reiner
This e-mail and any materials attached hereto, including, without limitation,
all content hereof and thereof (collectively, "XR Content") are confidential
and proprietary to XR Trading, LLC ("XR") and/or its affiliates, and are
protected by intellectual property laws. Without the prior written consent of
XR, the XR Content may not (i) be disclosed to any third party or (ii) be
reproduced or otherwise used by anyone other than current employees of XR or
its affiliates, on behalf of XR or its affiliates.
THE XR CONTENT IS PROVIDED AS IS, WITHOUT REPRESENTATIONS OR WARRANTIES OF ANY
KIND. TO THE MAXIMUM EXTENT PERMISSIBLE UNDER APPLICABLE LAW, XR HEREBY
DISCLAIMS ANY AND ALL WARRANTIES, EXPRESS AND IMPLIED, RELATING TO THE XR
CONTENT, AND NEITHER XR NOR ANY OF ITS AFFILIATES SHALL IN ANY EVENT BE LIABLE
FOR ANY DAMAGES OF ANY NATURE WHATSOEVER, INCLUDING, BUT NOT LIMITED TO,
DIRECT, INDIRECT, CONSEQUENTIAL, SPECIAL AND PUNITIVE DAMAGES, LOSS OF PROFITS
AND TRADING LOSSES, RESULTING FROM ANY PERSON'S USE OR RELIANCE UPON, OR
INABILITY TO USE, ANY XR CONTENT, EVEN IF XR IS ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES OR IF SUCH DAMAGES WERE FORESEEABLE.
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
