ot;< weston.p...@gmail.com >;
>
> 发件时间：2023/5/13 2:30
>
> 收件人："dev"< dev@arrow.apache.org >;
>
> 主题：Re: Reusing RecordBatch objects and their memory space
>
> I think there are perhaps various things being discussed here:
>
> * Reusing large bloc
apache.org >;
主题：Re: Reusing RecordBatch objects and their memory space
I think there are perhaps various things being discussed here:
* Reusing large blocks of memory
I don't think the memory pools actually provide this kind of reuse (e.g.
they aren't like "connection pools&
I think there are perhaps various things being discussed here:
* Reusing large blocks of memory
I don't think the memory pools actually provide this kind of reuse (e.g.
they aren't like "connection pools" or "thread pools"). I'm pretty sure,
when you allocate a new buffer on a pool, it always t
I can't find it anymore, but there is a quite old issue that made the same
observation: RecordBatch's heavy use of shared_ptr in C++ can lead to a lot of
overhead just calling destructors. That may be something to explore more (e.g.
I think someone had tried to "unbox" some of the fields in Reco
Hello,
I'm not sure if there are easy ways to avoid calling the destructors.
However, I would point out memory space reuse is handled through memory
pools; if you have one enabled it shouldn't be handing memory back to the
OS between each iteration.
Best,
Will Jones
On Fri, May 12, 2023 at 9:59
