Hi Dimitris, This sounds interesting. Is this for the C++ implementation? Could you go into a little bit more detail? How would this differ then using a MemoryPool implementation that always aligns to 4KB boundaries (instead of the 64 byte boundaries the default one does today [1])?
Thanks, Micah [1] https://github.com/apache/arrow/blob/master/cpp/src/arrow/memory_pool.cc#L40 On Wed, Mar 20, 2019 at 1:53 AM Dimitris Lekkas <dlekk...@gmail.com> wrote: > Hello folks, > > I am working at Inaccel where we utilize FPGAs to accelerate machine > learning workloads . Recently, we wanted to integrate our platform with > Arrow and we stumbled upon the non-alignment of data-buffers to page > boundaries (4KB). We implemented the option to supply per-column metadata > to page-align column vectors and we later memory mapped those columns > using native mmap() calls. Accelerators leverage page-alignment to i/o > efficiently, thus such an option might be an interesting addition to the > project. > In case you are interested, I will create a PR. > > Dimitris >
