Hi Angelo,
I think what might be happening here is that you have space characters in
your integer column, which are causing problems.
I created what could be a reproducible example of your problem at:
https://gist.github.com/thisisnic/af265166d5cd1ebce605cf3e478ee6d8
In short, can you try includ
Hi Everyone,
Thanks for the responses. I hope you are all well.
Hi Dewey. As to the problematic column error message: Invalid: Could not
open CSV input source 'folder/name.CSV': Invalid: In CSV column #30: Row
#5: CSV conversion error to int32: invalid value ''
I manually opened the csv and saw
In case it hasn't already been mentioned here, I wonder if manually setting
`schema()` would help. You're correct that the invalid value isn't
scientific notation (i.e., it's a blank string) so maybe that column should
be a string column instead. You could get the guessed schema from the
original o
Hi Angelo,
The original code with just `open_dataset()` works as it's created a
dataset without actually pulling the data into your R session. The
subsequent commands you tried (i.e. involving `collect()` read in the
files, resulting in an error when the data is read in.
It looks like there's an
Hi Jacob,
Thanks. To provide some specifics on my query:
1.which version of arrow are you running?
- 10.0.1
2. The error message provides an exact col,row position, have you checked
the value there?
Yes. It is int64. This is after running open_dataset without specifying
schema:
'''
arrow<-open_d
Hello Angelo,
just for completeness sake, which version of arrow are you running?
The error message provides an exact col,row position, have you checked the
value there? That could help pinpoint the cause of the error.
"Invalid: In CSV column #12: Row #580. CSV conversion error to int64:
invalid v
Hi,
I hope you are well. I am migrating to arrow from disk.frame in RStudio. I
am really impressed with how fast arrow compared to disk.frame but I need
help to solve some errors.
I wish to ask how I can resolve this error:
"CSV conversion error to int64: invalid value"
To give an idea
Hi ,
I hope you are well. I wish to ask how I can resolve this error:
"CSV conversion error to int64: invalid value"
To give an idea of my dataset. I have 4 csvs all placed in a local folder.
The code below worked when importing:
arrow<-open_dataset(
sources="csv location",
format="csv")
