Thanks for the discussion Weston and Gang! > I also somewhat disagree with this characterization. As long as the > producer and consumer agree on the type then there is no need for every > middleman (or even Arrow itself) to be aware of it. That's the entire goal > of extension types. IPC streams, the C data interface, Arrow flight, and > language bindings should all respect existing metadata and should _not_ > strip it out, even if they don't recognize it.
This is a great point and it's ideally how things should work. Now how do we get there? Should Parquet readers that output Arrow map Parquet logical types into "parquet.<logical_type>" extension types? If this requires non-trivial logic, to make the mapping unambiguous, are we implicitly specifying extension types at the conversion layer? > and so I will turn my vote into a +1 if needed for passage. Otherwise I'll > leave it as a half vote to discourage Arrow taking on the mantle of > blessing another format's types. Are you saying there should be super-project canonical extension types? :) Best, Rok On Sat, Nov 7, 2026 at 2:57 AM Gang Wu <[email protected]> wrote: > Thanks Weston for raising this! I have a different opinion. > > Canonical extension type for Parquet types like VARIANT、FILE and future > VECTOR brings better interoperability to different Parquet bindings in > Arrow. AFAIK, it would be a headache if different bindings define their > own custom extensions then we end up with different interpretations of > ARROW:schema serialization across bindings. > > Similarily, I was thinking of adding more canonical Parquet extension > types like GEOMETRY and GEOGRAPHY because I found it really tricky > when implementing Iceberg V3 types in iceberg-cpp where GeoArrow type > were not an option. > > On Sat, Nov 7, 2026 at 6:38 AM Weston Pace <[email protected]> wrote: > > > +0.5 > > > > I guess I'm a bit late to the party since we already have > > arrow.parquet.variant but it feels like parquet-specific types would just > > fit the definition for community extension types[1]. In other words, > > Parquet is free to define whatever extension types they want and there is > > no need for Arrow PMC approval in the same way that GeoArrow is free to > > define whatever extension types they want. > > > > > To ensure full interoperability, Arrow requires a matching canonical > > > in-memory extension type so that file fields do not lose their > > > semantic metadata during serialization across IPC streams, the C > > > Data Interface, Arrow Flight, and language bindings. > > > > I also somewhat disagree with this characterization. As long as the > > producer and consumer agree on the type then there is no need for every > > middleman (or even Arrow itself) to be aware of it. That's the entire > goal > > of extension types. IPC streams, the C data interface, Arrow flight, and > > language bindings should all respect existing metadata and should _not_ > > strip it out, even if they don't recognize it. > > > > That being said, I am happy for someone (even if it is Arrow and not > > Parquet) to bless a canonical extension type for any Parquet-specific > types > > and so I will turn my vote into a +1 if needed for passage. Otherwise > I'll > > leave it as a half vote to discourage Arrow taking on the mantle of > > blessing another format's types. > > > > [1] > > > > > https://arrow.apache.org/docs/format/CanonicalExtensions.html#community-extension-types > > > > On Fri, Nov 6, 2026 at 11:36 AM Mandukhai Alimaa < > > [email protected]> > > wrote: > > > > > Hi everyone, > > > > > > The FILE column type is being adopted across Parquet, Delta Lake, > > > Apache Iceberg, and Apache Spark so the broader ecosystem can natively > > > leverage structured file-level references. To ensure full > > > interoperability, Arrow requires a matching canonical in-memory > > > extension type so that file fields do not lose their semantic metadata > > > during serialization across IPC streams, the C Data Interface, Arrow > > > Flight, and language bindings. > > > > > > Following up on the discussion thread [1] and the subsequent format > > > change PR [2], we (@rok and I) would like to propose a formal vote to > > > add arrow.parquet.file as a canonical extension type to the Apache > > > Arrow format specification. This proposal already had reviews from > > > four different committers on GitHub and closely matches the Parquet > > > file logical type. > > > > > > [1] > > https://lists.apache.org/thread.html/5qw2k7rwstk0ol9ymds5qym81myznmpl > > > [2] https://github.com/apache/arrow/pull/51248 > > > [3] GitHub Issue: https://github.com/apache/arrow/issues/51143 > > > > > > The vote will be open for at least 72 hours. > > > > > > Please cast your vote: > > > [ ] +1 Accept this proposal > > > [ ] +0 > > > [ ] -1 Do not accept this proposal because... > > > > > > Thanks > > > > > >
