Help with zero-copy conversion of pyarrow table to pandas dataframe.

2018-09-28 Thread Bipin Mathew
Hello Everyone, I am just getting my feet wet with apache arrow and I am running into a bug or, more likely, simply misunderstanding the pyarrow api. I wrote out a four column, million row apache arrow table to shared memory and I am attempting to read it into a python dataframe. It is advert

Re: Help with zero-copy conversion of pyarrow table to pandas dataframe.

2018-09-28 Thread Bipin Mathew
2018 at 5:41 PM Wes McKinney wrote: > hi Abdul -- Plasma vs. a memory map on /dev/shm should have the same > semantics re: memory copying, so I don't believe using Plasma will > change the outcome > > - Wes > On Fri, Sep 28, 2018 at 5:38 PM Abdul Rahman > wrote: &g

Re: Help with zero-copy conversion of pyarrow table to pandas dataframe.

2018-10-03 Thread Bipin Mathew
"table" up there is a arrow::Table object. I create it columnwise using various arrow::[type]Builder functions. Is there anyway to actually even write the original table directly into shared memory? Any guidance on the proper way to do these things would be greatly appreciated. Regards,

[jira] [Created] (ARROW-1955) MSVC generates "attempting to reference a deleted function" during build.

2017-12-28 Thread Bipin Mathew (JIRA)
Bipin Mathew created ARROW-1955: --- Summary: MSVC generates "attempting to reference a deleted function" during build. Key: ARROW-1955 URL: https://issues.apache.org/jira/browse/ARROW-1955