[Discuss] C stream interface

[Antoine Pitrou]

Hello,

We are proposing an experimental C stream interface which builds on the
existing C data interface and is defined by a single C struct:

struct ArrowArrayStream {
  // Callbacks providing stream functionality
  int (*get_schema)(struct ArrowArrayStream*, struct ArrowSchema* out);
  int (*get_next)(struct ArrowArrayStream*, struct ArrowArray* out);
  const char* (*get_last_error)(struct ArrowArrayStream*);

  // Release callback
  void (*release)(struct ArrowArrayStream*);

  // Opaque producer-specific data
  void* private_data;
};

A PR is up and ready for review (including docs, a C++ import/export
implementation, and a simple Python binding):
https://github.com/apache/arrow/pull/8052

Here is the current documentation as of this writing (otherwise you can
also view the file from the PR diff view):
https://github.com/apache/arrow/blob/04c25d8cb8eb4c78066e651575aa8bfeef3ea542/docs/source/format/CStreamInterface.rst

Best regards

Antoine.