Emilio and I ran into some byte alignment issues last week. We're
generating data in the streaming format with the java lib, but the
javascript lib is failing to read it because some of the buffers don't
appear to be aligned.
Its not clear to us which and is implemented incorrectly - the spec
(https://arrow.apache.org/docs/memory_layout.html) says buffers should
be padded to 64 byte boundaries - does that extend to record batches in
the IPC formats?
The javascript implementation currently uses typed arrays to create
views for each buffer, which need to be aligned. We're looking into
using a DataView or a flatbuffers ByteBuffer to get around this issue
for now, but I'm wondering if this is a bug in the java implementation.
Brian
- 64-Byte Buffer Alignment bhulette
-
