Hi Micah, Thanks for the good suggestion. JNA seems like a good and reasonable tool for allocating large memory chunks.
How about we directly use Java UNSAFE API? It seems the allocateMemory API is also based on the malloc method of the native implementation [1]. Best, Liya Fan [1] http://hg.openjdk.java.net/jdk7/jdk7/hotspot/file/4fc084dac61e/src/share/vm/prims/unsafe.cpp On Sat, Jan 18, 2020 at 12:58 PM Micah Kornfield <[email protected]> wrote: > With the recently merged changes to the underlying ArrowBuf APIs to allow > 64-bit memory address spaces there is some follow-up work to actually > confirm it works. I opened a JIRA [1] to track this work. > > The main question is how to provide an allocator that supports larger > memory chunks. It appears the Netty API only takes an 32-bit integer for > array sizes. Doing a little bit of investigation it seems like JNA [2] > exposes a direct call to malloc of 64-bit integers [3]. > > The other options would seem to be rolling our own allocator via JNI. > > Is there anybody worked with JNA and can share experiences? > Is anyone familiar with other options? > > Thanks, > Micah > > [1] https://issues.apache.org/jira/browse/ARROW-7606 > [2] https://github.com/java-native-access/jna > [3] > > https://github.com/java-native-access/jna/blob/master/src/com/sun/jna/Native.java#L2265 >
