As far as I can tell, the alignment padding used in an IPC stream/file
isn't stored explicitly, and not really "inferrable", though maybe
technically possible if you calculated what bytes are *necessary* given a
buffer's data vs. what's actually stored.

Just wondering if this has been brought up at all to store explicitly; it
came up in the Julia implementation when considering "appending" record
batches to an IPC stream that has already been written to disk; we
originally thought we would need to match alignment used in previously
written record batches, but upon further reflection, it seems like
technically it wouldn't matter since all buffers have the exact byte counts
written anyway. Just wasn't sure if it would be breaking implicit
assumptions by consumers somewhere if they happened to get an IPC stream w/
record batches that mixed, for example, 8-byte and 64-byte alignments.

-Jacob

Reply via email to