Re: Re: Passing Arrow object across language

2018-07-22 Thread Masayuki Takahashi
Since this is an off heap memory, how could I make sure the memory will be > > released when necessary? > > > > thanks > > Yurui > > > > from Alimail macOS > > > > ------Original Mail ------ > > Sender:Wes McKinney > >

Re: Re: Passing Arrow object across language

2018-07-19 Thread Wes McKinney
er:Wes McKinney > Send Date:Tue Jul 17 02:09:51 2018 > Recipients: > Subject:Re: Passing Arrow object across language >> >> I discussed some of these things at a high level in my talk at SciPy >> 2018 last week >> >> >> https://www.slideshare.net/wes

Re: Re: Passing Arrow object across language

2018-07-16 Thread 周宇睿(闻拙)
the memory will be released when necessary? thanks Yurui from Alimail macOS --Original Mail -- Sender:Wes McKinney Send Date:Tue Jul 17 02:09:51 2018 Recipients: Subject:Re: Passing Arrow object across language I discussed some of these things at a high level in

Re: Passing Arrow object across language

2018-07-16 Thread Wes McKinney
I discussed some of these things at a high level in my talk at SciPy 2018 last week https://www.slideshare.net/wesm/apache-arrow-crosslanguage-development-platform-for-inmemory-data-105427919 On Mon, Jul 16, 2018 at 2:08 PM, Wes McKinney wrote: > hi Yurui, > > You can also share data structures

Re: Passing Arrow object across language

2018-07-16 Thread Wes McKinney
hi Yurui, You can also share data structures through JNI without using the IPC tools at all, which could require memory copying to produce the IPC messages. What you can do is obtain the memory addresses for the component buffers of an array (or vector, as called in Java) and construct the data s

Re: Passing Arrow object across language

2018-07-16 Thread Philipp Moritz
Hey Yuri, you can use the Arrow IPC mechanism to do this: - https://github.com/apache/arrow/blob/master/format/IPC.md - Python: https://arrow.apache.org/docs/python/ipc.html - C++: https://arrow.apache.org/docs/cpp/namespacearrow_1_1ipc.html - For Java, see the org.apache.arrow.vector.ipc namespa

Passing Arrow object across language

2018-07-16 Thread 周宇睿(闻拙)
Hi guys: I might miss something quite obviously. But how does Arrow passing objects across language? Let’s say I have a java program that invoke a c++ function via JNI, how does the c++ function pass an Arrow RecordBack object back to Java runtime without memory copy? Any advise would be appre