hi folks, In a prior mailing list thread from February [1] I brought up some work I'd done in C++ to create an API to define custom data types that can be embedded in built-in Arrow logical types. These are serialized through IPC by adding special fields to the `custom_metadata` member of Field in the Flatbuffers metadata [2]. The idea is that if an implementation does not understand the custom type, then they can still interact with the underlying data if need be, or pass on the extension metadata in subsequent IPC messages.
David Li has put up a WIP PR to implement this for Java [4], so to help the project move forward I think it's a good time to formalize this, and if there are disagreements to hash them out now. I have just opened a PR to the Arrow specification documents [3] that describes the current state of C++ and also the WIP Java PR. Any thought about this? If there is consensus about this solution approach then I can hold a vote. Thanks Wes [1]: https://lists.apache.org/thread.html/f1fc039471a8a9c06f2f9600296a20d4eb3fda379b23685f809118ee@%3Cdev.arrow.apache.org%3E [2]: https://github.com/apache/arrow/blob/master/format/Schema.fbs#L291 [3]: https://github.com/apache/arrow/pull/4332 [4]: https://github.com/apache/arrow/pull/4251
