I have a distributed database that returns query responses with a RecordBatchReader.
I'd like to distribute consumption of the query response by iterating the reader across a distributed compute environment (ray.io). I.e. round robin the calling read_next_batch over different nodes of the cluster, Is this possible? Ideally, with pyarrow, but I would be interested in other languages. Thanks!