With 1.0.0 imminent and corresponding forward compatibility guarantees, I think it is important that we provide ourselves with a mechanism for adding features within the existing flatbuffer schema that would allow older library versions to discover their inability to read the streams (flatbuffers does not appear to have a mechanism for exposing unknown fields). This can also potentially help standardize negotiations between clients and servers for flight.
I put together a PR with a proposal on how to do this [1]. The PR adds a new enum "Feature", and adds a list of Feature to the Schema object. As a strawman, I added dictionary replacement and buffer compression which don't yet have wide implementation support. We could leave these out, as clients should already be able to detect those changes as of the last release. Other potential options: 1. Add the enum to the wrapper message instead of the schema. 2. Use a Union and Tables for richer representations. (I did not choose this approach because it makes forward compatibility more awkward if we realize there is a missing field in the table and need to add it later). Thoughts? Thanks, Micah [1] https://github.com/apache/arrow/pull/7502
