Hey Wes, Since all our logic for writing and reading the data is written in C, we need to be get access to the memory in C as well. I think it makes sense then for me to work on creating a C++ wrapper around Plasma that exports what is necessary in C. Should that be in Arrow GLib?
Best, Saul On Tue, Mar 6, 2018 at 11:49 PM Wes McKinney <wesmck...@gmail.com> wrote: > hi Saul -- I think the easiest solution here is the buffer/memoryview > protocol. You won't have to touch the Cython or C++ API from pyarrow > if you do this. > > You can interact with a Buffer object like any other Python object > implementing the buffer protocol. See numpy.frombuffer as an example > of a function that interacts with such objects. I would suggest adding > a method to xnd method for this. > > If you need to interact with Plasma from C then things will be more > complicated -- Robert or Philipp should be able to advise in this > case. > > - Wes > > On Tue, Mar 6, 2018 at 4:55 PM, Saul Shanabrook <s.shanabr...@gmail.com> > wrote: > > Hey Wes, > > > > I don't have much experience doing C + Python + Cython development, so I > am > > probably missing something obvious, but reading the Cython docs, > > < > https://cython.readthedocs.io/en/latest/src/userguide/external_C_code.html#public-declarations > > > > it > > seems like I can only access types marked as public from C code. When I > > compile arrow locally, I do get some C++ headers for the plasma code, > but I > > don't think I can use them from C code either. > > > > Best, > > Saul > > > > > > > > On Tue, Mar 6, 2018 at 3:12 PM Wes McKinney <wesmck...@gmail.com> wrote: > > > >> hi Saul, > >> > >> Are you able to use the buffer/memoryview protocol? Instances of > >> pyarrow.Buffer, like PlasmaBuffer, support this > >> > >> > https://github.com/apache/arrow/blob/master/python/pyarrow/plasma.pyx#L182 > >> > >> - Wes > >> > >> On Tue, Mar 6, 2018 at 3:09 PM, Saul Shanabrook <s.shanabr...@gmail.com > > > >> wrote: > >> > I am trying to use the Plasma store to back xnd objects. Xnd ( > >> > https://xnd.readthedocs.io/en/latest/xnd/index.html) is a container > >> library > >> > in C that has Python bindings. I would like to get a pointer to the > >> > allocated memory after creating or get an object in Plasma. I see that > >> this > >> > is supported in the C++ API ( > >> > > >> > https://arrow.apache.org/docs/cpp/classplasma_1_1_plasma_client.html#ac18ab9cc792c620a97a3dcb165e0ecd7 > >> ) > >> > but not in the python API (as far as I can tell). Is it possible to > use > >> the > >> > C++ Plasma API from a C project? If not, would it make sense to expose > >> > pointer access on the Python API using capsules > >> > https://docs.python.org/3.6/c-api/capsule.html > >> > <https://docs.python.org/3.6/c-api/capsule.html#capsules>? > >> >
