Sure, I've created https://issues.apache.org/jira/browse/ARROW-5255.
PR: https://github.com/apache/arrow/pull/4251 I'm not sure if what I'm doing with my Vector subclass is quite right, but we'd especially like this in Java, so happy to work through any feedback. Also, as part of this discussion, I think the original C++ implementation noted that this metadata would not round-trip through Pandas. We would definitely like that feature if possible - maybe column-level metadata could be saved under a special field in the dataframe-level Pandas metadata? Best, David On 5/3/19, Wes McKinney <wesmck...@gmail.com> wrote: > hi David -- would you like to open a PR and corresponding JIRA issue > for discussion? We might want to hold a vote to formalize the > extension type mechanism (and to fix the metadata names -- I agree > that having an ARROW namespace would be better than what we are doing > now) > > On Thu, May 2, 2019 at 7:02 AM David Li <li.david...@gmail.com> wrote: >> >> Re: Java support, I've sketched out an implementation: >> https://github.com/lihalite/arrow/pull/2 >> >> On 5/1/19, Micah Kornfield <emkornfi...@gmail.com> wrote: >> >> >> >> I'm awaiting community feedback about the approach to implementing >> >> extension types, whether the approach that I've used (using the >> >> following keys in custom_metadata [1]) is the one that we want to use >> >> longer-term. This certainly seems like a good time to have that >> >> discussion. If there is consensus then we can document it formally in >> >> the specification documents, and we probably will want to hold a vote >> >> to ensure that we are in agreement. >> >> >> > >> > Please let me know if this is best on a separate thread. I think I >> > would >> > feel more comfortable finalizing this if we had a few more examples >> > exercising the functionality. Inet, seems like a complicated enough >> > use-case for modeling which would make it a good use-case (It seems like >> > it >> > might involve a struct/union?). I also presume we will need a Java >> > implementation, before we finalize anything? >> > >> > A small amount of bikeshedding on key names: We should probably take a >> > namespace reservation approach for custom metadata in Schema.fbs [1]. >> > In >> > this regard I have a small preference for something reserving all >> > metadata >> > with something like "ARROW:<reserved_arrow_key>" or "ARROW." (not an >> > underscore, and I'm open to different capitalization.) This seems to be >> > a >> > similar approach to how avro reserves metadata keys [2]. >> > >> > [1] >> > https://github.com/apache/arrow/blob/b8aeb79e94a5a507aeec55d0b6c6bf5d7f0100b2/format/Schema.fbs#L264 >> > [2] https://avro.apache.org/docs/1.8.1/spec.html >> > >
