>From a datasets / Acero perspective I have been thinking about this in the back of my mind for a while and decided to write my thoughts down in a document. I will send it in a separate email.
On Tue, Nov 8, 2022 at 9:53 AM Micah Kornfield <emkornfi...@gmail.com> wrote: > > Hi Matthew, > Could you give some more specifics about what language/component you are > using. In general, Arrow at a specification level doesn't deal with schema > evolution. Is this in regard to Datasets or a different component? > > Thanks, > Micah > > On Mon, Nov 7, 2022 at 5:06 PM Matthew Scanlon < > matthew.scan...@exosfinancial.com> wrote: > > > Good afternoon, I wanted to reach out and open a dialog about structs, the > > evolution of them in schemas, and if support for such a feature is on the > > road map or a hard pass for the arrow team. > > > > Currently, it appears structs support removing a field, but will there be > > support for adding fields later on? Are there any recommended patterns for > > supporting such a field. For example, if a field foo is a struct with > > sub_fields A, B and then later field C gets added, the old data can not be > > loaded using the new schema. > > > > Thank you. > > > > Matthew Scanlon > > > > -- > > > > > > Broker-Dealer services offered through Exos Securities LLC, member of > > SIPC <http://www.sipc.org/> / FINRA <http://www.finra.org/> / > > BrokerCheck > > <https://brokercheck.finra.org/>/ 2022 Exos, inc. For important > > disclosures, click here > > <https://www.exosfinancial.com/general-disclosures>. > > > > > > > >
