This seems like a bug to me; I would not expect this to fail. It's too bad it didn't get fixed in time for 0.12
On Thu, Jan 10, 2019 at 4:20 PM Bryan Cutler <cutl...@gmail.com> wrote: > > Hi All, > > I have a question about using pyarrow.Array.from_pandas with the safe flag > set to True. When the Pandas data contains integers and NULL values, it > will get changed to a floating point dtype and then if the type is casted > back to an integer in Arrow, it will raise an error "ArrowInvalid: Floating > point value truncated". Is this the expected behavior? I'm guessing it > doesn't look at the actual values, just what type is being converted. Is > there a way around this specific error besides setting safe to False? Here > is a concise example: > > >>> pa.Array.from_pandas(pd.Series([1, None]), type=pa.int32(), safe=True) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "pyarrow/array.pxi", line 474, in pyarrow.lib.Array.from_pandas > File "pyarrow/array.pxi", line 169, in pyarrow.lib.array > File "pyarrow/array.pxi", line 69, in pyarrow.lib._ndarray_to_array > File "pyarrow/error.pxi", line 81, in pyarrow.lib.check_status > pyarrow.lib.ArrowInvalid: Floating point value truncated > > I came across this issue in https://github.com/apache/spark/pull/22807, > specifically withi this discussion > https://github.com/apache/spark/pull/22807#discussion_r246859417. > > Thanks! > Bryan
