Hi Vignesh,
Is this about a specific Arrow implementation? If so, can you state
which one? (also, please update the e-mail subject line to include that
information)
Regards
Antoine.
Le 05/02/2026 à 11:15, Vignesh Siva a écrit :
Hi all,
I’m currently working on issue #6586, which discusses supporting Arrow C
Stream interfaces that yield a stream of generic ArrowArray, rather than
assuming StructArray / RecordBatch.
At the moment, ArrowArrayStreamReader assumes the stream produces
StructArray and eagerly constructs RecordBatch. One approach I’ve been
exploring is introducing a separate stream reader that yields ArrayRef
directly, deriving the DataType from the stream’s FFI_ArrowSchema, while
keeping the existing RecordBatch-based reader unchanged for backward
compatibility.
Before moving further, I wanted to ask for guidance on the preferred API
direction:
Would maintainers favor a separate ArrayRef-based reader for generic
streams, or would a more unified abstraction (optionally producing
RecordBatch when the underlying array is Struct) be more in line with the
project’s long-term design?
Any feedback or design guidance would be greatly appreciated.
Thanks,
Vignesh
