I've created a document[1] that both describes the general idea of schema evolution as well as my best guess at how it should work. This is written from an Acero / datasets perspective but the information should be generally applicable / accessible.
I am doing some work in the scanner to enable an extension point for schema evolution but I am probably not going to be implementing this in the next few months. However, I would like to get to it at some point in the next year or possibly guide some others on implementation. So I'd appreciate any feedback or opinions on the doc before too much work gets done. [1] https://docs.google.com/document/d/1iWPm8wf6CwJzX6MIFPNzwaaVIjPr1sKRXw1qNr4Jq6c/edit#heading=h.94f33whmotx5
