Wes McKinney created ARROW-8250:
-----------------------------------
Summary: [C++] Add "random access" / slice read API to
RecordBatchFileReader
Key: ARROW-8250
URL: https://issues.apache.org/jira/browse/ARROW-8250
Project: Apache Arrow
Issue Type: Improvement
Components: C++
Reporter: Wes McKinney
Fix For: 1.0.0
If you want to read a small section of a file, it is not possible to easily
determine the relevant record batches that need "rehydrating".
I would propose the following:
* A way to cheaply read (and cache, so this doesn't have to be done multiple
times) all the RecordBatch metadata without deserializing the record batch data
structures themselves
* Based on the metadata you can then determine the range of batches that need
to be rehydrated and then sliced accordingly to produce the Table of interest
This functionality can be lifted into the Feather read APIs also
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
