Jonathan mercier created ARROW-8273:
---------------------------------------
Summary: Fail to convert an integer list (arrow) to pandas
Key: ARROW-8273
URL: https://issues.apache.org/jira/browse/ARROW-8273
Project: Apache Arrow
Issue Type: Bug
Components: Python
Reporter: Jonathan mercier
Dear,
I have an arrow table and one of columns as an arrow list(size=2)
When I try to convert the table to pandas I got an ArrowNotImplementedError
error
below minimal case:
{code:java}
from pyarrow import Schema, Table, int64, list_, schema, array
fields = [('foo', list_(int64(), 2),)]
sc = schema(fields)
foo_column = [[1,2], [3,4]]
columns = [foo_column]
a_table = Table.from_arrays(arrays=columns, schema=sc)
df = a_table.to_pandas()
---------------------------------------------------------------------------
ArrowNotImplementedError Traceback (most recent call last)
<ipython-input-13-87e390bb8b12> in <module>
----> 1 df2 = a_table.to_pandas()
~/somewhere/foo/venv/lib64/python3.7/site-packages/pyarrow/array.pxi in
pyarrow.lib._PandasConvertible.to_pandas()
~/somewhere/foo/venv/lib64/python3.7/site-packages/pyarrow/table.pxi in
pyarrow.lib.Table._to_pandas()
~/somewhere/foo/venv/lib64/python3.7/site-packages/pyarrow/pandas_compat.py in
table_to_blockmanager(options, table, categories, ignore_metadata, types_mapper)
764 _check_data_column_metadata_consistency(all_columns)
765 columns = _deserialize_column_index(table, all_columns, column_indexes)
--> 766 blocks = _table_to_blocks(options, table, categories,
ext_columns_dtypes)
767
768 axes = [columns, index]
~/somewhere/foo/venv/lib64/python3.7/site-packages/pyarrow/pandas_compat.py in
_table_to_blocks(options, block_table, categories, extension_columns)
1099 columns = block_table.column_names
1100 result = pa.lib.table_to_blocks(options, block_table, categories,
-> 1101 list(extension_columns.keys()))
1102 return [_reconstruct_block(item, columns, extension_columns)
1103 for item in result]
~/somewhere/foo/venv/lib64/python3.7/site-packages/pyarrow/table.pxi in
pyarrow.lib.table_to_blocks()
~/somewhere/foo/venv/lib64/python3.7/site-packages/pyarrow/error.pxi in
pyarrow.lib.check_status()
ArrowNotImplementedError: No known equivalent Pandas block for Arrow data of
type fixed_size_list<item: int64>[2] is known
{code}
Maybe I need to convert python list (column) to array
Thanks for your help
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
