Re: Using Plasma with xnd

; now. > >>> > > >>> > @Saul, you will want to create a C wrapper for arrow::Buffer so you > >>> > can manage memory lifetime from C. Depending on the kind of Buffer > you > >>> > have, it may not be as simple as calling free(...) on its

Re: Using Plasma with xnd

internal >>> > data. >>> > >>> > - Wes >>> > >>> > On Tue, Mar 27, 2018 at 10:31 AM, Kouhei Sutou >>> wrote: >>> > > Hi, >>> > > >>> > > It may be wrong but you may need

Re: Using Plasma with xnd

ling free(...) on its internal >> > data. >> > >> > - Wes >> > >> > On Tue, Mar 27, 2018 at 10:31 AM, Kouhei Sutou >> wrote: >> > > Hi, >> > > >> > > It may be wrong but you may need

Re: Using Plasma with xnd

es > > > > On Tue, Mar 27, 2018 at 10:31 AM, Kouhei Sutou > wrote: > > > Hi, > > > > > > It may be wrong but you may need to add -DNDEBUG flag to C > > > compiler. > > > > > > > > > Thanks, > > > -- &

Re: Using Plasma with xnd

10:31 AM, Kouhei Sutou wrote: > > Hi, > > > > It may be wrong but you may need to add -DNDEBUG flag to C > > compiler. > > > > > > Thanks, > > -- > > kou > > > > In > > "Re: Using Plasma with xnd" on Tue, 27 Mar 201

Re: Using Plasma with xnd

Tue, Mar 27, 2018 at 10:31 AM, Kouhei Sutou wrote: > Hi, > > It may be wrong but you may need to add -DNDEBUG flag to C > compiler. > > > Thanks, > -- > kou > > In > "Re: Using Plasma with xnd" on Tue, 27 Mar 2018 13:41:49 +, >

Re: Using Plasma with xnd

Hi, It may be wrong but you may need to add -DNDEBUG flag to C compiler. Thanks, -- kou In "Re: Using Plasma with xnd" on Tue, 27 Mar 2018 13:41:49 +, Saul Shanabrook wrote: > Ah, I see I can use garrow_buffer_get_mutable_data from c_glib. > > On Tue, Mar 27, 2

Re: Using Plasma with xnd

Ah, I see I can use garrow_buffer_get_mutable_data from c_glib. On Tue, Mar 27, 2018 at 9:29 AM Saul Shanabrook wrote: > I am in the process of changing over my C wrapper to work with Arrow > 0.9.0. One change is that the plasma APIs now use arrow::Buffer instead of > of raw pointers, in Create

Re: Using Plasma with xnd

I am in the process of changing over my C wrapper to work with Arrow 0.9.0. One change is that the plasma APIs now use arrow::Buffer instead of of raw pointers, in Create and Get. In my C wrapper for Create, I need to get the mutable data pointer from the arrow::Buffer, after it is created. Here i

Re: Using Plasma with xnd

Where are you going to make Plasma requests, in Python or in C? * If in Python, use the buffer protocol to access the memory in C https://docs.python.org/3/c-api/buffer.html#bufferobjects * If in C, then I would suggest adding extern "C" APIs directly in libplasma.so so you can connect to the Plas

Re: Using Plasma with xnd

Hey Wes, Since all our logic for writing and reading the data is written in C, we need to be get access to the memory in C as well. I think it makes sense then for me to work on creating a C++ wrapper around Plasma that exports what is necessary in C. Should that be in Arrow GLib? Best, Saul On

Re: Using Plasma with xnd

hi Saul -- I think the easiest solution here is the buffer/memoryview protocol. You won't have to touch the Cython or C++ API from pyarrow if you do this. You can interact with a Buffer object like any other Python object implementing the buffer protocol. See numpy.frombuffer as an example of a fu

Re: Using Plasma with xnd

Hey Wes, I don't have much experience doing C + Python + Cython development, so I am probably missing something obvious, but reading the Cython docs, it seems like I can only access types marked as pub

Re: Using Plasma with xnd

hi Saul, Are you able to use the buffer/memoryview protocol? Instances of pyarrow.Buffer, like PlasmaBuffer, support this https://github.com/apache/arrow/blob/master/python/pyarrow/plasma.pyx#L182 - Wes On Tue, Mar 6, 2018 at 3:09 PM, Saul Shanabrook wrote: > I am trying to use the Plasma stor