> Are you saying there should be super-project canonical extension types? :)
No. I am suggesting the Parquet PMC publish an extension-types.md document in [1] that describes what the extension type definitions are. This would be exactly like [2]. This lets the Parquet team bless whatever unambiguous definition they like. Anyone wanting to consume custom Parquet types must use those definitions. The Parquet team is then in complete control and doesn't need to seek the Arrow team's blessing. The Arrow team can then create a link back to that document in [3] in much the same way we do for GeoArrow. For example, the Substrait type system does not name all Arrow types. in Arrow, we publish [4] which describes Substrait extension types to map to the existing Arrow types. Or, as another example, Lance's blob type is an Arrow extension type defined in [5]. Consumers of Lance custom types consult the Lance spec for those types' definitions. > Similarily, I was thinking of adding more canonical Parquet extension > types like GEOMETRY and GEOGRAPHY because I found it really tricky > when implementing Iceberg V3 types in iceberg-cpp where GeoArrow type > were not an option. If Iceberg has its own set of types (separate from Parquet and GeoArrow) then Iceberg can also maintain an extension-types.md document right? [1] https://github.com/apache/parquet-format/ [2] https://github.com/geoarrow/geoarrow/blob/main/extension-types.md [3] https://arrow.apache.org/docs/format/CanonicalExtensions.html#community-extension-types [4] https://github.com/apache/arrow/blob/main/format/substrait/extension_types.yaml [5] https://github.com/lance-format/lance/blob/main/docs/src/format/table/blob.md#table-schema On Sat, Nov 7, 2026 at 7:00 AM Rok Mihevc <[email protected]> wrote: > Thanks for the discussion Weston and Gang! > > > I also somewhat disagree with this characterization. As long as the > > producer and consumer agree on the type then there is no need for every > > middleman (or even Arrow itself) to be aware of it. That's the entire > goal > > of extension types. IPC streams, the C data interface, Arrow flight, and > > language bindings should all respect existing metadata and should _not_ > > strip it out, even if they don't recognize it. > > This is a great point and it's ideally how things should work. Now how do > we get there? Should Parquet readers that output Arrow map Parquet logical > types into "parquet.<logical_type>" extension types? If this requires > non-trivial logic, to make the mapping unambiguous, are we implicitly > specifying extension types at the conversion layer? > > > and so I will turn my vote into a +1 if needed for passage. Otherwise > I'll > > leave it as a half vote to discourage Arrow taking on the mantle of > > blessing another format's types. > > Are you saying there should be super-project canonical extension types? :) > > > Best, > Rok > > On Sat, Nov 7, 2026 at 2:57 AM Gang Wu <[email protected]> wrote: > > > Thanks Weston for raising this! I have a different opinion. > > > > Canonical extension type for Parquet types like VARIANT、FILE and future > > VECTOR brings better interoperability to different Parquet bindings in > > Arrow. AFAIK, it would be a headache if different bindings define their > > own custom extensions then we end up with different interpretations of > > ARROW:schema serialization across bindings. > > > > Similarily, I was thinking of adding more canonical Parquet extension > > types like GEOMETRY and GEOGRAPHY because I found it really tricky > > when implementing Iceberg V3 types in iceberg-cpp where GeoArrow type > > were not an option. > > > > On Sat, Nov 7, 2026 at 6:38 AM Weston Pace <[email protected]> > wrote: > > > > > +0.5 > > > > > > I guess I'm a bit late to the party since we already have > > > arrow.parquet.variant but it feels like parquet-specific types would > just > > > fit the definition for community extension types[1]. In other words, > > > Parquet is free to define whatever extension types they want and there > is > > > no need for Arrow PMC approval in the same way that GeoArrow is free to > > > define whatever extension types they want. > > > > > > > To ensure full interoperability, Arrow requires a matching canonical > > > > in-memory extension type so that file fields do not lose their > > > > semantic metadata during serialization across IPC streams, the C > > > > Data Interface, Arrow Flight, and language bindings. > > > > > > I also somewhat disagree with this characterization. As long as the > > > producer and consumer agree on the type then there is no need for every > > > middleman (or even Arrow itself) to be aware of it. That's the entire > > goal > > > of extension types. IPC streams, the C data interface, Arrow flight, > and > > > language bindings should all respect existing metadata and should _not_ > > > strip it out, even if they don't recognize it. > > > > > > That being said, I am happy for someone (even if it is Arrow and not > > > Parquet) to bless a canonical extension type for any Parquet-specific > > types > > > and so I will turn my vote into a +1 if needed for passage. Otherwise > > I'll > > > leave it as a half vote to discourage Arrow taking on the mantle of > > > blessing another format's types. > > > > > > [1] > > > > > > > > > https://arrow.apache.org/docs/format/CanonicalExtensions.html#community-extension-types > > > > > > On Fri, Nov 6, 2026 at 11:36 AM Mandukhai Alimaa < > > > [email protected]> > > > wrote: > > > > > > > Hi everyone, > > > > > > > > The FILE column type is being adopted across Parquet, Delta Lake, > > > > Apache Iceberg, and Apache Spark so the broader ecosystem can > natively > > > > leverage structured file-level references. To ensure full > > > > interoperability, Arrow requires a matching canonical in-memory > > > > extension type so that file fields do not lose their semantic > metadata > > > > during serialization across IPC streams, the C Data Interface, Arrow > > > > Flight, and language bindings. > > > > > > > > Following up on the discussion thread [1] and the subsequent format > > > > change PR [2], we (@rok and I) would like to propose a formal vote to > > > > add arrow.parquet.file as a canonical extension type to the Apache > > > > Arrow format specification. This proposal already had reviews from > > > > four different committers on GitHub and closely matches the Parquet > > > > file logical type. > > > > > > > > [1] > > > https://lists.apache.org/thread.html/5qw2k7rwstk0ol9ymds5qym81myznmpl > > > > [2] https://github.com/apache/arrow/pull/51248 > > > > [3] GitHub Issue: https://github.com/apache/arrow/issues/51143 > > > > > > > > The vote will be open for at least 72 hours. > > > > > > > > Please cast your vote: > > > > [ ] +1 Accept this proposal > > > > [ ] +0 > > > > [ ] -1 Do not accept this proposal because... > > > > > > > > Thanks > > > > > > > > > >
