Re: [Python] Converting Python Schema Object to Java Schema Object

2022-05-21 Thread Antoine Pitrou
Hello Srinivas, No, there is not. Also, the pyarrow.jvm is a bit limited currently, there are plans to rewrite it to make it more general (you may want to help contribute if you feel interested and skilled enough): https://issues.apache.org/jira/browse/ARROW-14319 Regards Antoine. Le 2

[Python] Converting Python Schema Object to Java Schema Object

2022-05-21 Thread Srinivas Lade
Hi dev@arrow, The pyarrow.jvm library contains the schema() function which can convert a schema object from the arrow java library to a python object that pyarrow can use. Is there a function that does the opposite: convert a pyarrow schema to a java arrow schema?

Re: [C++] Control flow and scheduling in C++ Engine operators / exec nodes

2022-05-21 Thread Supun Kamburugamuve
Thanks, Weston. Now I understand a little deeper. About point 2. I have previously seen the pipeline prioritization you've described with both sides running simultaneously. My experience was not good with that approach and one side's first approach was much better. About these points, if you are

[C++] parquet::arrow::WriteTable process exited with code 0177

2022-05-21 Thread Rares Vernica
Hello, We have a plugin that writes both Arrow and Parquet format files. We are experiencing an issue when using Parquet format, while Arrow format works just fine. More exactly, the process crashes in parquet::arrow::WriteTable. Using gdb we identified the line when the process crashes https://g

Re: [C++] parquet::arrow::WriteTable process exited with code 0177

2022-05-21 Thread Micah Kornfield
Hi Rares, Based on the stack trace, I'm not seeing any Arrow/Parquet related frames. Two things come to mind: 1. This seems like it might be heap corruption (see https://stackoverflow.com/questions/10578315/segmentation-fault-in-dl-runtime-resolve). Try running using ASAN or valgrind. 2. (Less l