[
https://issues.apache.org/jira/browse/ARROW-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lawrence Chan updated ARROW-2296:
---------------------------------
Description:
Maybe I'm overlooking something, but I don't see something on the API surface
to get the number of rows in a arrow file without reading all the record
batches. This is useful when we want to read into contiguous buffers, because
it allows us to allocate the right sizes up front.
I'd like to propose that we add `num_rows` as a field in the file footer so
it's easy to query without reading the whole file.
Meanwhile, before we get that added to the official format fbs, it would be
nice to have a method that iterates over the record batch headers and sums up
the lengths without reading the actual record batch body.
was:
Maybe I'm overlooking something, but I don't see something on the API surface
to get the number of rows in a arrow file without reading all the record
batches.
I'd like to propose that we add `num_rows` as a field in the file footer so
it's easy to query without reading the whole file.
Meanwhile, before we get that added to the official format fbs, it would be
nice to have a method that iterates over the record batch headers and sums up
the lengths without reading the actual record batch body.
> Add num_rows to file footer
> ---------------------------
>
> Key: ARROW-2296
> URL: https://issues.apache.org/jira/browse/ARROW-2296
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++, Format
> Reporter: Lawrence Chan
> Priority: Minor
>
> Maybe I'm overlooking something, but I don't see something on the API surface
> to get the number of rows in a arrow file without reading all the record
> batches. This is useful when we want to read into contiguous buffers, because
> it allows us to allocate the right sizes up front.
> I'd like to propose that we add `num_rows` as a field in the file footer so
> it's easy to query without reading the whole file.
> Meanwhile, before we get that added to the official format fbs, it would be
> nice to have a method that iterates over the record batch headers and sums up
> the lengths without reading the actual record batch body.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
