Hi all,
I am helping resolve this GH issue [1] with this PR [2], where user wants
to use `CRecordBatch.column_data()` method from Cython to access the
underlying `CArrayData`
but `column_data()` is not exposed in `CRecordBatch`. There is a workaround
to access the `CArrayData` [3]. Nevertheless, m
Le 25/08/2021 à 17:27, Joris Van den Bossche a écrit :
https://github.com/rapidsai/cudf/blob/be25a30ca20f3135f341c51b36cb075b376d5def/python/cudf/cudf/_lib/cpp/io/types.pxd#L9
Here they are doing `from pyarrow.includes.libarrow cimport
CRandomAccessFile` (CRandomAccessFile is the cython equiva
On Wed, 25 Aug 2021 at 17:21, Antoine Pitrou wrote:
>
> Le 25/08/2021 à 17:12, Joris Van den Bossche a écrit :
> > One example of consumer of our Cython API is cudf (
> > https://github.com/rapidsai/cudf).
> > I am not very familiar with the package itself, but browsing its code, I
> > see that t
Le 25/08/2021 à 17:12, Joris Van den Bossche a écrit :
One example of consumer of our Cython API is cudf (
https://github.com/rapidsai/cudf).
I am not very familiar with the package itself, but browsing its code, I
see that they do for example cimport RecordBatchReader (
https://github.com/rapi
Le 25/08/2021 à 17:17, Keith Kraus a écrit :
If I remember correctly the reason cuDF interacts with the Cython code for
IPC stuff is that in the past the existing IPC machinery in Arrow didn't
work correctly with GPU memory. If that is fixed I think there's a case to
remove this code entirely f
If I remember correctly the reason cuDF interacts with the Cython code for
IPC stuff is that in the past the existing IPC machinery in Arrow didn't
work correctly with GPU memory. If that is fixed I think there's a case to
remove this code entirely from cuDF and instruct users to use the higher
lev
One example of consumer of our Cython API is cudf (
https://github.com/rapidsai/cudf).
I am not very familiar with the package itself, but browsing its code, I
see that they do for example cimport RecordBatchReader (
https://github.com/rapidsai/cudf/blob/f6d31fa95d9b8d8658301438d0f9ba22a1c131aa/pyt
Le 20/08/2021 à 12:24, Alessandro Molina a écrit :
We could argue that only what was documented explicitly should be
considered "public" and everything else can be changed, but our
documentation seems to be unclear on this point. It lists some functions
that should be considered our explicit a
Given we didn't get much opinions on this one, I will propose we move
forward with merging the open PR that moves ipc cython implementation and
discover if we receive any open issue because projects out there were
relying on it.
It seems that ipc is a low risk module from that point of view and wil
