If this were accomplished at the application level, how would it work
with the IPC formats? I'd think you'd need to have two separate files
(or streams), since array 1 and array 2 will be different lengths.
Perhaps that could be an argument for making span a core logical type?
Brian
On 05/02/2018 03:34 PM, Antoine Pitrou wrote:
On Wed, 2 May 2018 10:12:37 -0400
Wes McKinney <wesmck...@gmail.com> wrote:
It sounds like the "span" type could be implemented as a composite of
multiple Arrow arrays / schemas:
array 1 (data)
any schema
array 2 (view)
struct <
start: int64,
stop: int64
Unless I'm missing something, this feels like an application-level
concern rather than something that needs to be addressed in the
columnar format / metadata.
Well, couldn't the same theoretically be said about list arrays?
In the end, I suppose it all depends whether there's enough demand to
make it a core logical type inside Arrow, rather than something people
write custom code for in their application.
Regards
Antoine.
