[
https://issues.apache.org/jira/browse/ARROW-11324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17268603#comment-17268603
]
Andrew Lamb edited comment on ARROW-11324 at 1/20/21, 2:45 PM:
---------------------------------------------------------------
[~m18e] I can try and take a look at fixing this -- do you have a reproducer
(e.g. the input file) easily at hand?
was (Author: alamb):
[~m18e] I can try and take a look at fixing this -- do you have a reproducer
easily at hand?
> [Rust] Querying datetime data in DataFusion with an embedded timezone always
> fails
> ----------------------------------------------------------------------------------
>
> Key: ARROW-11324
> URL: https://issues.apache.org/jira/browse/ARROW-11324
> Project: Apache Arrow
> Issue Type: Bug
> Components: Rust - DataFusion
> Reporter: Max Burke
> Priority: Blocker
>
> We have a number (~ hundreds of thousands) of Parquet files that have
> embedded Arrow schemas in them that have time-valued columns with the type
> DateTime(TimeUnit::Nanosecond, Some("UTC")).
>
> One of the changes in the Arrow 2 -> 3 working window was to make the Parquet
> loader prefer the Arrow schema compared to the one generated from the
> columns.
>
> But because DataFusion has the timezone field of the DateTime variant
> hardcoded as None, we can't load any of our data after this upgrade; we get
> errors like:
> {{SELECT * FROM parquet_table WHERE ("timestamp" >=
> to_timestamp('2010-03-24T13:00:00.000000Z') AND "timestamp" <=
> to_timestamp('2010-03-25T00:00:00.000000Z')) ORDER BY timestamp ASC NULLS
> LAST;}}
> {{Plan("\'Timestamp(Nanosecond, Some(\"UTC\")) >= Timestamp(Nanosecond,
> None)\' can\'t be evaluated because there isn\'t a common type to coerce the
> types to")}}
>
> Any ideas/thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
