Re: Understanding "shared" memory implications

2016-03-19 Thread Zhe Zhang
I have similar concerns as Todd stated below. With an mmap-based approach, we are treating shared memory objects like files. This brings in all filesystem related considerations like ACL and lifecycle mgmt. Stepping back a little, the shared-memory work isn't really specific to Arrow. A few questi

Re: I setup a slack team to have a live channel to discuss Arrow

2016-03-11 Thread Zhe Zhang
Please invite z...@apache.org. Thanks. On Fri, Mar 11, 2016 at 1:25 PM Yiannis Gkoufas wrote: > Hi there, would love an invite as well. > johngou...@gmail.com > > Thanks!! > > On 11 March 2016 at 21:11, Kyle White wrote: > > > Please send an invite to 'kjw...@gmail.com'. > > > > Thanks, > > Kyl

Re: Question about mutability

2016-02-24 Thread Zhe Zhang
tions themselves are worried about exposing their own memory spaces. > > > On Wed, Feb 24, 2016 at 2:17 PM, Andrew Brust < > andrew.br...@bluebadgeinsights.com> wrote: > > > Hmm...that's not exactly how Jaques described things to me when he > briefed > > me

Re: Question about mutability

2016-02-24 Thread Zhe Zhang
rote: > > > Hmm...that's not exactly how Jaques described things to me when he > briefed me on Arrow ahead of the announcement. > > -Original Message- > From: Zhe Zhang [mailto:z...@apache.org] > Sent: Wednesday, February 24, 2016 2:08 PM > To: dev@arrow.apach

Re: Question about mutability

2016-02-24 Thread Zhe Zhang
I don't think one application/process's memory space will be made available to other applications/processes. It's fundamentally hard for processes to share their address spaces. IIUC, with Arrow, when application A shares data with application B, the data is still duplicated in the memory spaces o

Re: Need help for contribution to Arrow

2016-02-22 Thread Zhe Zhang
Many perspectives in the Hadoop "how to contribute" doc apply to Apache projects in general: https://wiki.apache.org/hadoop/HowToContribute I'll leave to active Arrow contributors to add Arrow-specific rules. I think creating a JIRA ticket is always a good starting point (assuming you already have