This has been discussed several times over the years, see [1]. I think if someone wanted to pick that up and carry it over the finish line, it would be welcomed.
That said, some Arrow implementations provide support for serializing/deserializing a schema separately from data (e.g. PyArrow/Arrow C++/Arrow Java), which would also work. [1]: https://github.com/apache/arrow/pull/7110 On Fri, Nov 3, 2023, at 11:28, Benjamin MacDonald Schmidt wrote: > Hi all, > > Is there any standard or general way to serialize table schemas, field > definitions, or type definitions to text (presumably as JSON)? > > For example, I can create a thing like `pa.field("coords", > pa.struct([pa.field("x", pa.float32())]))`. Then I want to pass information > about that field to the javascript Arrow implementation to check if it has > a "z" column in the struct before deciding to render some UI element in the > browser > > The simplest way I can see to do this is to wrap the one field into a table > schema, serialize that schema to IPC, encode the bytes to base64, ship and > deserialize, decode the schema in js, pop out the one field, and examine > it. While I do this in some cases, it gets cumbersome: it sure would be > helpful to have a convention like `{"name": "coords", "type": "struct", > "content": [{"name": "x", "type": "float32"}]}` that I could borrow, if > anyone's invented it. > > Thanks to everyone at the project for all your work! > > Ben > > > --- > Ben Schmidt > VP Information Design, Nomic AI > atlas.nomic.ai
