Like Pandas, in the R package we store R-specific metadata in the schema; this lets us preserve all R class and other attributes when round-tripping data to Feather or Parquet. These are not stored in a way that is readable or useful to other implementations, partly by design. If there were type information/metadata that was meaningful across libraries, it would make sense to define (and document) an extension type.
Some discussion here: https://arrow.apache.org/docs/r/articles/arrow.html#r-object-attributes Neal On Thu, Apr 8, 2021 at 10:52 AM Micah Kornfield <emkornfi...@gmail.com> wrote: > 1. Do the standard libraries handle that metadata key automatically ? > > C++, Python and Java have facilities to support them automatically > (extensions needs to register themselves), I'm not sure about other > languages. > > 2. Are there other standard or best practice metadata keys that either > people generally use or the standard libraries expect? > > I think there are some things related to Pandas types and parquet data that > are propagated automatically through metadata. Rust was looking into > including file sources but I'm not sure that is being externalized. There > might be other things, hopefully people will chime in. > > On Thu, Apr 8, 2021 at 10:47 AM Michael Lavina <michael.lav...@factset.com > > > wrote: > > > Hey Team, > > > > I noticed that under extension type the Arrow docs specifically call out > > to `ARROW:extension:name` and `ARROW:extension:metadata` as > > recommended/reserved metadata keys to handle extension types. > > > > 2 quick questions > > > > > > 1. Do the standard libraries handle that metadata key automatically ? > > 2. Are there other standard or best practice metadata keys that either > > people generally use or the standard libraries expect? > > > > Thank you, > > Michael > > >
