Hi devs, I am creating a pyarrow.Array from a pandas.Series and when specifying the data type, if the value will be truncated an error is raised. For example:
import pyarrow as pa import pandas as pd s = pd.Series([pd.Timestamp(1)]) arr = pa.Array.from_pandas(s, type=pa.timestamp('us')) /home/bryan/miniconda2/lib/python2.7/site-packages/pyarrow/error.pxi in pyarrow.lib.check_status (/arrow/python/build/temp.linux-x86_64-2.7/lib.cxx:8270)() 75 message = frombytes(status.message()) 76 if status.IsInvalid(): ---> 77 raise ArrowInvalid(message) 78 elif status.IsIOError(): 79 raise ArrowIOError(message) ArrowInvalid: Casting from timestamp[ns] to timestamp[us] would lose data: 1 If I do the casting after creating the pyarrow.Array like pa.Array.from_pandas(s).cast(pa.timestamp('us'), safe=False) then it is fine. My question is does pyarrow.Array.from_pandas only use safe casting or is there a way to specify that truncation is ok? Thanks, Bryan