I am just
> not seeing it.
>
> -Michael
> From: Weston Pace
> Date: Friday, February 26, 2021 at 9:34 PM
> To: dev@arrow.apache.org
> Cc: Michael Lavina
> Subject: Re: [JS] Exploring usage of apache arrow at my company for complex
> table rendering
> I used Arrow
ervers so hopefully this works.
>> > >
>> > > That’s exactly what I was thinking of doing is to create a standard
>> > > metadata schema to built on top of Apache Arrow with some predefined user
>> > > types.
>> > >
>> &g
very easy I am
> just not seeing it.
>
> -Michael
> From: Weston Pace
> Date: Friday, February 26, 2021 at 9:34 PM
> To: dev@arrow.apache.org
> Cc: Michael Lavina
> Subject: Re: [JS] Exploring usage of apache arrow at my company for
> complex table rendering
> I used Arrow
ache.org
Cc: Michael Lavina
Subject: Re: [JS] Exploring usage of apache arrow at my company for complex
table rendering
I used Arrow for this purpose in the past. I don't have much to add
but just a few thoughts off the top of my head...
* The line between data and metadata can be blurry
g the metadata and that could be
> > > anything but maybe like you said we should be creating a separate standard
> > > entirely for defining the schema to render tables instead of defining it
> > > within Arrow.
> > >
> > > Does it defeat the value of Arrow if a
eat the value of Arrow if are sending the data using buffers
> > and stream and a giant string of stringified metadata when I could maybe
> > define the metadata in protobuf binary separately.
> >
> > In addition, I was curious with all these visualization tools has someone
a, relationship between columns and
> hidden information.
>
> -Michael
>
> From: Neal Richardson
> Date: Friday, February 26, 2021 at 1:38 PM
> To: dev
> Subject: Re: [JS] Exploring usage of apache arrow at my company for
> complex table rendering
> The Arrow IPC
standard metadata for arrow to help with rendering. Stuff
like how to denote grouping of data, relationship between columns and hidden
information.
-Michael
From: Neal Richardson
Date: Friday, February 26, 2021 at 1:38 PM
To: dev
Subject: Re: [JS] Exploring usage of apache arrow at my company
The Arrow IPC specification allows for custom metadata in both the Schema
and the individual Fields:
https://arrow.apache.org/docs/format/Columnar.html#schema-message
Might that work for you? Another alternative would be to track your
metadata in a separate object outside of the Arrow data.
Neal
Hello Everyone,
Some background. My name is Michael and I work at FactSet, which if you use
Arrow you may have heard because one of our architects did a talk on using
Arrow and Dremio.
https://hello.dremio.com/eliminate-data-transfer-bottlenecks-with-apache-arrow-flight.html?utm_medium=social-fr
10 matches
Mail list logo
