Eric Erhardt created ARROW-5908:
-----------------------------------

             Summary: [C#] ArrowStreamWriter doesn't align buffers to 8 bytes
                 Key: ARROW-5908
                 URL: https://issues.apache.org/jira/browse/ARROW-5908
             Project: Apache Arrow
          Issue Type: Improvement
          Components: C#
            Reporter: Eric Erhardt
            Assignee: Eric Erhardt


When writing RecordBatches using ArrowStreamWriter, if the ArrowBuffers being 
written aren't all 8 byte aligned, the serialized RecordBatch won't conform to 
the Arrow specification. This leads to other languages' readers to throw an 
error when reading Arrow streams written by the C# writer.

For example, if reading the stream from Python or C++, an error is raised here: 

[https://github.com/apache/arrow/blob/f77c3427ca801597b572fb197b92b0133269049b/cpp/src/arrow/ipc/reader.cc#L107-L110]

A similar error is raised when Java tries to read the stream.

We should be ensuring that the buffers being written to the stream are padded 
to 8 bytes, no matter their length, as specified in 
[https://arrow.apache.org/docs/format/Layout.html#requirements-goals-and-non-goals]

 
{quote} * It is required to have all the contiguous memory buffers in an IPC 
payload aligned at 8-byte boundaries. In other words, each buffer must start at 
an aligned 8-byte offset. Additionally, each buffer should be padded to a 
multiple of 8 bytes.{quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to