Worth noting: the c data interface explicitly forbids adding new members [1] to its structs, so simply adding ArrowArray::buffer_sizes is not viable.
[1] https://github.com/bkietz/arrow/blob/0afb739a16672483b69894c6fe3f5ece5cfc79d8/docs/source/format/CDataInterface.rst?plain=1#L984-L986 On Wed, Oct 25, 2023 at 12:36 PM Benjamin Kietzman <bengil...@gmail.com> wrote: > Hello all, > > The C ABI does not store buffer lengths explicitly, which presents a > problem for Utf8View since buffer lengths are not trivially extractable > from other data in the array. A potential solution is to store the lengths > in an extra buffer after the variadic data buffers. I've adopted this > approach in my (currently draft) PR [1] to add c++ library import/export > for Utf8VIew, but I thought this warranted raising on the ML in case anyone > has a better idea. > > Sincerely, > Ben Kietzman > > [1] > https://github.com/bkietz/arrow/compare/37710-cxx-impl-string-view..36099-string-view-c-abi#diff-3907fc8e8c9fa4ed7268f6baa5b919e8677fb99947b7384a9f8f001174ab66eaR549 >
