[
https://issues.apache.org/jira/browse/ARROW-3656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669208#comment-16669208
]
Antoine Pitrou commented on ARROW-3656:
---------------------------------------
It's not a good idea to infer only from the first row, because of nulls or
other false positives.
Indeed there's no absolutely correct answer, but type inference should make a
best effort to give satisfying results on real-world data. Right now we don't
allow passing a schema to impose fixed types, though that's something we should
add.
> [C++] Allow whitespace in numeric CSV fields
> --------------------------------------------
>
> Key: ARROW-3656
> URL: https://issues.apache.org/jira/browse/ARROW-3656
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Affects Versions: 0.11.0
> Reporter: Antoine Pitrou
> Assignee: Antoine Pitrou
> Priority: Major
>
> Pandas allows whitespace before and after numbers in CSV files, but Arrow
> doesn't:
> {code:python}
> >>> s = b"a,b,c\n12 , 34 , 56\n"
> >>> pd.read_csv(io.BytesIO(s))
> a b c
> 0 12 34 56
> >>> csv.read_csv(io.BytesIO(s)).to_pandas()
> a b c
> 0 b'12 ' b' 34 ' b' 56'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
