I've discovered while working on ARROW-9399 that it is very difficult
with the Flatbuffers API in C++ to detect a MetadataVersion [1] that
is higher than the current version.
For example, suppose that 3 or 4 years from now we move from version
V5 to version V6. The generated Flatbuffers code looks like this
org::apache::arrow::flatbuf::MetadataVersion version() const {
return
static_cast<org::apache::arrow::flatbuf::MetadataVersion>(GetField<int16_t>(VT_VERSION,
0));
}
That static cast is currently undefined behavior.
One way to deal with this would be to add placeholder future versions
(e.g. V6 and V7).
Another backward-and-forward-compatible option would be to return the
version as a short (int16_t) rather than the enum value, which is
subject to this brittleness.
Either way unfortunately I think adding forward compatibility checks
is out of scope for 1.0.0 and the risk is low since we don't
anticipate bumping the version anytime soon.
Thanks,
Wes
[1]: https://github.com/apache/arrow/blob/master/format/Schema.fbs#L22
