issue
https://github.com/apache/arrow/issues/36100... If any one can highlight or
suggest a way to overcome this problem will be helpful. Appreciate your
help.!
Regards,
Alex
On Wed, Jun 14, 2023 at 9:35 PM Jerald Alex wrote:
> Hi Experts,
>
> Pyarrow *Table.from_pylist* does not relea
Hi Experts,
Pyarrow *Table.from_pylist* does not release memory until the program
terminates. I created a sample script to highlight the issue. I have also
tried setting up `pa.jemalloc_set_decay_ms(0)` but it didn't help much.
Could you please check this and let me know if there are potential iss
CSV tend to need more configuration and
> tuning. For example, setting the delimiter, skipping some header rows,
> etc. Parquet is pretty self-describing and you would only need to use the
> fragment_scan_options if, for example, you need to decryption or custom
> control over
Hi Experts,
I have been using dataset.scanner to read the data with specific filter
conditions and batch_size of 1000 to read the data.
ds.scanner(filter=pc.field('a') != 3, batch_size=1000).to_batches()
I would like to know if it is possible to skip the specific set of batches,
for example, the
t; # child 0, entries: struct not null
> # child 0, key: string not null
> # child 1, value: string
> # ----
> # id: [[1,2]]
> # names:
>
> [[keys:["first_name","last_name"]values:["Tyler","Brady"],keys:["first_name","last_name"
Any inputs on this please?
On Tue, May 2, 2023 at 10:03 AM Jerald Alex wrote:
> Hi Experts,
>
> Can anyone please highlight if it is possible to cast struct to map type?
>
> I tried the following but it seems to be producing an error as below.
>
> pyarrow.lib.Arr
Hi Experts,
Can anyone please highlight if it is possible to cast struct to map type?
I tried the following but it seems to be producing an error as below.
pyarrow.lib.ArrowNotImplementedError: Unsupported cast from
struct to map using function cast_map
Note: Snippet is just an example to show
