Hi all, I’m currently working on issue #6586, which discusses supporting Arrow C Stream interfaces that yield a stream of generic ArrowArray, rather than assuming StructArray / RecordBatch.
At the moment, ArrowArrayStreamReader assumes the stream produces StructArray and eagerly constructs RecordBatch. One approach I’ve been exploring is introducing a separate stream reader that yields ArrayRef directly, deriving the DataType from the stream’s FFI_ArrowSchema, while keeping the existing RecordBatch-based reader unchanged for backward compatibility. Before moving further, I wanted to ask for guidance on the preferred API direction: Would maintainers favor a separate ArrayRef-based reader for generic streams, or would a more unified abstraction (optionally producing RecordBatch when the underlying array is Struct) be more in line with the project’s long-term design? Any feedback or design guidance would be greatly appreciated. Thanks, Vignesh
