[
https://issues.apache.org/jira/browse/ARROW-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396530#comment-16396530
]
ASF GitHub Bot commented on ARROW-2122:
---------------------------------------
wesm commented on issue #1707: ARROW-2122: [Python] Pyarrow fails to serialize
dataframe with timestamp.
URL: https://github.com/apache/arrow/pull/1707#issuecomment-372543159
This needs a little more scrutiny (cc @jreback @cpcloud @pitrou) before we
commit to something for 0.9.0 we might have to break later. I'm going to move
this issue off 0.9.0 so we aren't blocking
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> [Python] Pyarrow fails to serialize dataframe with timestamp.
> -------------------------------------------------------------
>
> Key: ARROW-2122
> URL: https://issues.apache.org/jira/browse/ARROW-2122
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Reporter: Robert Nishihara
> Assignee: Albert Shieh
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.10.0
>
>
> The bug can be reproduced as follows.
> {code:java}
> import pyarrow as pa
> import pandas as pd
> df = pd.DataFrame({'A': [pd.Timestamp('2012-11-11 00:00:00+01:00'), pd.NaT]})
> s = pa.serialize(df).to_buffer()
> new_df = pa.deserialize(s) # this fails{code}
> The last line fails with
> {code:java}
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "serialization.pxi", line 441, in pyarrow.lib.deserialize
> File "serialization.pxi", line 404, in pyarrow.lib.deserialize_from
> File "serialization.pxi", line 257, in
> pyarrow.lib.SerializedPyObject.deserialize
> File "serialization.pxi", line 174, in
> pyarrow.lib.SerializationContext._deserialize_callback
> File "/home/ubuntu/arrow/python/pyarrow/serialization.py", line 77, in
> _deserialize_pandas_dataframe
> return pdcompat.serialized_dict_to_dataframe(data)
> File "/home/ubuntu/arrow/python/pyarrow/pandas_compat.py", line 446, in
> serialized_dict_to_dataframe
> for block in data['blocks']]
> File "/home/ubuntu/arrow/python/pyarrow/pandas_compat.py", line 446, in
> <listcomp>
> for block in data['blocks']]
> File "/home/ubuntu/arrow/python/pyarrow/pandas_compat.py", line 466, in
> _reconstruct_block
> dtype = _make_datetimetz(item['timezone'])
> File "/home/ubuntu/arrow/python/pyarrow/pandas_compat.py", line 481, in
> _make_datetimetz
> return DatetimeTZDtype('ns', tz=tz)
> File
> "/home/ubuntu/anaconda3/lib/python3.5/site-packages/pandas/core/dtypes/dtypes.py",
> line 409, in __new__
> raise ValueError("DatetimeTZDtype constructor must have a tz "
> ValueError: DatetimeTZDtype constructor must have a tz supplied{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
