On Thu, Feb 25, 2016 at 12:05 PM, Henry Robinson wrote:
> The way I'm thinking about is that someone upstream makes a Kudu-specific
> request, but as part of that request provides a descriptor of a shared
> ring-buffer. Reading Arrow batches from and writing to that buffer is part
> of a simple st
reed,
> >> >> > > >
> >> >> > > > I thought the whole purpose was to share the memory space
> (using
> >> >> > possibly
> >> >> > > > unsafe operations like ByteBuffers) so that it could be
> directly
> >> >> shared
> >> >
gt; > > datastore that one application can expose and share with others
>> (e.g.
>> >> > In
>> >> > > > memory structure is constructed from a series of parquet files,
>> >> > somehow,
>> >> > > > then Spark pulls it i
datastore that one application can expose and share with others
> (e.g.
> >> > In
> >> > > > memory structure is constructed from a series of parquet files,
> >> > somehow,
> >> > > > then Spark pulls it in, does some computations, exposes a d
mputations, exposes a data set,
>> > > > etc...).
>> > > >
>> > > > If you are leaving the allocation of the memory to the applications
>> and
>> > > > underneath the memory is being allocated using direct bytebuffers, I
>> > > ca
tally hard- especially if the
> > > > applications themselves are worried about exposing their own memory
> > > spaces.
> > > >
> > > >
> > > > On Wed, Feb 24, 2016 at 2:17 PM, Andrew Brust <
> > > > andrew.br...@bluebadgein
4, 2016 at 2:17 PM, Andrew Brust <
> > > andrew.br...@bluebadgeinsights.com> wrote:
> > >
> > > > Hmm...that's not exactly how Jaques described things to me when he
> > > briefed
> > > > me on Arrow ahead of the announcement.
> > >
..@bluebadgeinsights.com> wrote:
> >
> > > Hmm...that's not exactly how Jaques described things to me when he
> > briefed
> > > me on Arrow ahead of the announcement.
> > >
> > > -Original Message-
> > > From: Zhe Zhang [mailto:z...@apa
on Arrow ahead of the announcement.
> >
> > -Original Message-
> > From: Zhe Zhang [mailto:z...@apache.org]
> > Sent: Wednesday, February 24, 2016 2:08 PM
> > To: dev@arrow.apache.org
> > Subject: Re: Question about mutability
> >
> > I don&#x
>
> >
> >
> >
> > On Wed, Feb 24, 2016 at 11:17 AM, Andrew Brust <
> > andrew.br...@bluebadgeinsights.com> wrote:
> >
> > > Hmm...that's not exactly how Jaques described things to me when he
> > briefed
> > > me on Arrow ahead
ch
taking turns working with the data, in a cumulative fashion.
-Original Message-
From: Michael D. Coon [mailto:mdco...@yahoo.com.INVALID]
Sent: Wednesday, February 24, 2016 2:21 PM
To: dev@arrow.apache.org
Subject: Re: Question about mutability
I had the same understanding as Corey
x27;s not exactly how Jaques described things to me when he
> briefed
> > me on Arrow ahead of the announcement.
> >
> > -Original Message-
> > From: Zhe Zhang [mailto:z...@apache.org]
> > Sent: Wednesday, February 24, 2016 2:08 PM
> > To: dev@arrow.apac
ust <
andrew.br...@bluebadgeinsights.com> wrote:
> Hmm...that's not exactly how Jaques described things to me when he briefed
> me on Arrow ahead of the announcement.
>
> -Original Message-
> From: Zhe Zhang [mailto:z...@apache.org]
> Sent: Wednesday, February 24, 2016 2:08 PM
> T
efed
> me on Arrow ahead of the announcement.
>
> -Original Message-
> From: Zhe Zhang [mailto:z...@apache.org]
> Sent: Wednesday, February 24, 2016 2:08 PM
> To: dev@arrow.apache.org
> Subject: Re: Question about mutability
>
> I don't think one application/
rote:
>
>
> Hmm...that's not exactly how Jaques described things to me when he
> briefed me on Arrow ahead of the announcement.
>
> -Original Message-
> From: Zhe Zhang [mailto:z...@apache.org]
> Sent: Wednesday, February 24, 2016 2:08 PM
> To: dev@arrow.apach
rg
Subject: Re: Question about mutability
I don't think one application/process's memory space will be made available to
other applications/processes. It's fundamentally hard for processes to share
their address spaces.
IIUC, with Arrow, when application A shares data with applicat
Hmm...that's not exactly how Jaques described things to me when he briefed me
on Arrow ahead of the announcement.
-Original Message-
From: Zhe Zhang [mailto:z...@apache.org]
Sent: Wednesday, February 24, 2016 2:08 PM
To: dev@arrow.apache.org
Subject: Re: Question about mutabili
I don't think one application/process's memory space will be made available
to other applications/processes. It's fundamentally hard for processes to
share their address spaces.
IIUC, with Arrow, when application A shares data with application B, the
data is still duplicated in the memory spaces o
18 matches
Mail list logo
