Hello,

I've recently started working with extension types as part of our project
and I was surprised to discover that extension types are required to pack
all of their own metadata into a single string value of the
"ARROW:extension:metadata" key.

In turn this then means we have to endure arbitrary unstructured /
hard-to-validate strings with custom encodings (e.g. JSON inside
flatbuffer) when dealing with extensions.

Can anyone provide some context on the rationale for this design decision?

Given that we already have (1) a perfectly good metadata keyvalue store
already in place, and (2) established recommendations for
namespaced scoping of keys, why would we not just use that to store the
metadata for the extension. For example:

"ARROW:extension:name": "myorg.myextension",
"myorg:myextension:meta1": "value1",
"myorg:myextension:meta2": "value2",

Thanks for any insights,
Jeremy

Reply via email to