>>> it as
>>> >>> a tiered storage for sort spilling (before hitting the disk), or
>>> should we
>>> >>> use it for caching?
>>> >>> Gary: my 2 cents, the netty library looks not yet provide a elegant
>>> switch
>>> >
>
> Thanks,
> Yanping
>
>
> -Original Message-
> From: Jacques Nadeau [mailto:jacq...@apache.org]
> Sent: Wednesday, March 30, 2016 7:34 PM
> To: dev@arrow.apache.org
> Cc: d...@mnemonic.incubator.apache.org; d...@mnemonic.apache.org
> Subject: Re: A Proposal Apach
rg; d...@mnemonic.apache.org
Subject: Re: A Proposal Apache Incubator Mnemonic as an alternative infra. for
Apache Arrow
Yup. Will do.
The discussion today was limited to "let's meet".
On Wed, Mar 30, 2016 at 7:13 PM, P. Taylor Goetz wrote:
> +1
>
> Discussions should be s
ry-like spaces
> instead
> >>> of part logics of class PooledByteBufAllocatorL.
> >>> Regarding the sorting, I think it is a typical case of random access to
> >>> the data, we should avoid spilling as much as possible.
> >>> my 2 c
>>> my 2 cents, the performance could be
>>> all in off-heap if possible > mnemonic used as cache > all in mnemonic
>>> using NVMe/disk > off-heap + spilling
>>> the code simplicity would be
>>> all in off-heap if possible > all in mnemonic using
l in off-heap if possible > all in mnemonic using NVMe/disk > mnemonic
>> used as cache > off-heap + spilling
>>
>> the reason why the mode "mnemonic used as cache + spilling" probably
>> unnecessary is mnemonic could provide nearly equivalent capacity of d
anks.
> Gary.
>
>
> -----Original Message-
>
> From: Jacques Nadeau [mailto:jacq...@apache.org]
>
> Sent: Tuesday, March 29, 2016 8:05 AM
>
> To: <mailto:dev@arrow.apache.org> dev@arrow.apache.org dev@arrow.apache.org>
>
> Subject: Re: A Proposal Apac
t; probably
unnecessary is mnemonic could provide nearly equivalent capacity of disk.
Thanks.
Gary.
-Original Message-
From: Jacques Nadeau [mailto:jacq...@apache.org]
Sent: Tuesday, March 29, 2016 8:05 AM
To: <mailto:dev@arrow.apache.org>
dev@arrow.apache.org<mailto:dev@arrow
pache.org]
Sent: Tuesday, March 29, 2016 8:05 AM
To: dev@arrow.apache.org <mailto:dev@arrow.apache.org>
Subject: Re: A Proposal Apache Incubator Mnemonic as an alternative
infra. for Apache Arrow
This is super cool. A couple of questions:
- Right now you are using unpooled persisten
This is super cool. A couple of questions:
- Right now you are using unpooled persistent memory. Does that make sense
or does chunking make more sense?
- What do you think is the right way to transition back and forth between
persistent and ephemeral memory? What do you think will be the first
pat
Hello,
We have created a patch for Apache Arrow to leverage Apache incubator
Mnemonic as an alternative infra. for underlying memory resources
allocation, you can find it as below forked repo.
https://github.com/NonVolatileComputing/arrow
By this way, Apache Arrow could take some structur
11 matches
Mail list logo
