RE: RE: Re: Pyarrow minimal build for lambda layers

2023-11-29 Thread Steven Handerson
Oh - important detail, the directions that I was following are in https://arrow.apache.org/docs/developers/python.html . Steve On 2023/11/27 18:38:55 Akshara Sadheesh wrote: > Thank you so much for your reply Raul! So I did run the build using the > build_venv.sh file. The issue was I think I d

RE: RE: Re: Pyarrow minimal build for lambda layers

2023-11-29 Thread Steven Handerson
I have a similar problem - using a conda build, following the pyarrow build instructions. It works fine on the build machine, but building and installing the wheel ends up missing some libraries (libutf8, for starters). I’m kind of a newbie in this regard, could someone spell out how you can do

RE: Re: Pyarrow minimal build for lambda layers

2023-11-27 Thread Akshara Sadheesh
Thank you so much for your reply Raul! So I did run the build using the build_venv.sh file. The issue was I think I did not copy over the libarrow.so files from my docker container in the `root/dist/lib` directory. I have added them onto `arrow/python/pyarrow`. After the build finished I copie

Re: Pyarrow minimal build for lambda layers

2023-11-22 Thread Raúl Cumplido
Hi Shara, The example dockerfile installs the base requirements for Ubuntu but then we use the build_venv.sh (or build_conda.sh) to build the Arrow CPP library and then pyarrow [1]. >From the error it seems you did not build Arrow CPP as libarrow.so can't be found. Can you try following the recip

Pyarrow minimal build for lambda layers

2023-11-21 Thread Akshara Sadheesh
Hi, I have been trying to use the minimal_build for python with the provided examples Dockerfile.ubuntu for my lambda layers since it has a 250 MB limit. I am able to run the build and generate a pyarrow library. However, the library does not contain any shared .so files. When in use, it says: `"