Hi Pearu,
Your approach seems ok on the principle. Feel free to open a PR once you have a basic API defined and exposed, so that we can iterate on that. Regards Antoine. Le 29/08/2018 à 17:11, Pearu Peterson a écrit : > Hi Wes, > > Thanks for the hints! > > I have created a very minimal GPU support to pyarrow, see > > > https://github.com/apache/arrow/compare/master...Quansight:pearu-cuda-pyarrow?expand=1 > > This includes: > 1. introducing libarrow_gpu that is built into pyarrow.lib_gpu extension > module, currently only CudaBuffer is exposed > 2. updating cmake configuration files to detect libarrow_gpu.{so,a} > libraries > 3. updating setup.py with new options --with{,out}-arrow-gpu and > PYARROW_BUILD_ARROW_GPU env variable support. > > Let me know if something in my approach needs to be changed. > > Regarding packaging: > 1. FYI, there is a cudatoolkit-dev package in conda-forge: > https://github.com/conda-forge/cudatoolkit-dev-feedstock > > 2. One approach is to introduce new conda packages arrow-cpp-gpu and > pyarrow-gpu that are build using -DARROW_GPU and --with-arrow-gpu, > respectively. > > Best regards, > Pearu > > On Wed, Aug 29, 2018 at 12:32 AM, Wes McKinney <wesmck...@gmail.com> wrote: > >> hi Pearu, >> >> Sounds good to me; I had wanted to do it already but got caught up >> working on other things. You'll probably want to create >> libarrow_gpu.pxd since this is an optional library. Otherwise it >> should be reasonably straightforward. >> >> some relevant issues >> >> - https://issues.apache.org/jira/browse/ARROW-1424 >> - https://issues.apache.org/jira/browse/ARROW-2512 >> >> If we wanted to ship Python wheels and conda packages with CUDA >> support built in, what would we have to do (currently these binaries >> are produced in Circle CI, Travis CI, and Appveyor)? >> >> - Wes >> On Tue, Aug 28, 2018 at 4:34 PM Pearu Peterson >> <pearu.peter...@quansight.com> wrote: >>> >>> Hi, >>> >>> Is there any interest in implementing CudaBuffer support in pyarrow? Are >>> there any obstacles that would prevent to do this? >>> >>> I would be willing to give it a try. >>> >>> As I understand, the entry point for this task would be to patch >>> pyarrow/includes/libarrow.pxd . Any hints for what to take into account >> are >>> appreciated. >>> >>> Best regards, >>> Pearu >> >
