Hi Alex,
passing the schema to from_pylist() method on the Table should work for
your example (not sure if it solves your initial problem?)
import pyarrow as pa
table_schema = pa.schema([pa.field("id", pa.int32()),
pa.field("names", pa.map_(pa.string(), pa.string()))])
table_data = [{"id": 1,"names": {"first_name": "Tyler", "last_name":"Brady"
}},
{"id": 2,"names": {"first_name": "Walsh", "last_name": "Weaver"}}]
pa.Table.from_pylist(table_data, schema=table_schema)
# pyarrow.Table
# id: int32
# names: map<string, string>
# child 0, entries: struct<key: string not null, value: string> not null
# child 0, key: string not null
# child 1, value: string
# ----
# id: [[1,2]]
# names:
[[keys:["first_name","last_name"]values:["Tyler","Brady"],keys:["first_name","last_name"]values:["Walsh","Weaver"]]]
Best, Alenka
On Wed, May 3, 2023 at 9:13 AM Jerald Alex <vminf...@gmail.com> wrote:
> Any inputs on this please?
>
> On Tue, May 2, 2023 at 10:03 AM Jerald Alex <vminf...@gmail.com> wrote:
>
> > Hi Experts,
> >
> > Can anyone please highlight if it is possible to cast struct to map type?
> >
> > I tried the following but it seems to be producing an error as below.
> >
> > pyarrow.lib.ArrowNotImplementedError: Unsupported cast from
> > struct<first_name: string, last_name: string> to map using function
> cast_map
> >
> > Note: Snippet is just an example to show the problem.
> >
> > Code Snippet:
> >
> > table_schema = pa.schema([pa.field("id", pa.int32()), pa.field("names",
> > pa.map_(pa.string(), pa.string()))])
> >
> > table_data = [{"id": 1,"names": {"first_name": "Tyler", "last_name":
> > "Brady"}},
> > {"id": 2,"names": {"first_name": "Walsh", "last_name": "Weaver"}}]
> >
> > tbl = pa.Table.from_pylist(table_data)
> > print(tbl)
> > print(tbl.cast(table_schema))
> > print(tbl)
> >
> > Error :
> >
> > id: int64
> > names: struct<first_name: string, last_name: string>
> > child 0, first_name: string
> > child 1, last_name: string
> > ----
> > id: [[1,2]]
> > names: [
> > -- is_valid: all not null
> > -- child 0 type: string
> > ["Tyler","Walsh"]
> > -- child 1 type: string
> > ["Brady","Weaver"]]
> > Traceback (most recent call last):
> > File "/Users/
> >
> infant.a...@cognitedata.com/Documents/Github/HubOcean/demo/pyarrow_types.py
> ",
> > line 220, in <module>
> > print(tbl.cast(table_schema))
> > File "pyarrow/table.pxi", line 3489, in pyarrow.lib.Table.cast
> > File "pyarrow/table.pxi", line 523, in pyarrow.lib.ChunkedArray.cast
> > File "/Users/
> >
> infant.a...@cognitedata.com/Library/Caches/pypoetry/virtualenvs/demo-LzMA3Hsd-py3.10/lib/python3.10/site-packages/pyarrow/compute.py
> ",
> > line 391, in cast
> > return call_function("cast", [arr], options)
> > File "pyarrow/_compute.pyx", line 560, in
> pyarrow._compute.call_function
> > File "pyarrow/_compute.pyx", line 355, in
> pyarrow._compute.Function.call
> > File "pyarrow/error.pxi", line 144, in
> > pyarrow.lib.pyarrow_internal_check_status
> > File "pyarrow/error.pxi", line 121, in pyarrow.lib.check_status
> > pyarrow.lib.ArrowNotImplementedError: Unsupported cast from
> > struct<first_name: string, last_name: string> to map using function
> cast_map
> >
> > Regards,
> > Alex Vincent
> >
>
