+1 to use "parquet.file" as well. On Wed, Sep 2, 2026 at 7:29 PM Matt Topol <[email protected]> wrote:
> We ended up doing "arrow.parquet.variant" so should we do > "arrow.parquet.file" for symmetry? > > On Wed, Sep 2, 2026, 10:20 PM Gang Wu <[email protected]> wrote: > > > +1 for "parquet.file" > > > > I would expect it to be very similar to how we define "parquet.variant". > > > > On Thu, Sep 3, 2026 at 5:22 AM Rok Mihevc <[email protected]> wrote: > > > > > > Shouldn't the extension name be `parquet.file`? It's really a direct > > > > mapping of the Parquet FILE type, not something we decided to > > > > standardize independently. > > > > > > parquet.file sounds good in that it makes it clear where the > > specification > > > comes from. > > > > > > > Also I wonder if it's the Arrow community's role to standardize such > > > > extension types, or if we should let the Parquet community do it? > > > > > > We have non-arrow namespaced types in geoarrow [1], but they are > > specified > > > in geoarrow repo. Are you proposing that? > > > > > > Parquet already specified the file type, we'd just be adding a mirror > > type > > > in Arrow to make handling it practical. > > > > > > [1] https://github.com/geoarrow/geoarrow/blob/main/extension-types.md > > > > > > Rok > > > > > > On Wed, Sep 2, 2026 at 11:05 PM Antoine Pitrou <[email protected]> > > wrote: > > > > > > > > > > > Shouldn't the extension name be `parquet.file`? It's really a direct > > > > mapping of the Parquet FILE type, not something we decided to > > > > standardize independently. > > > > > > > > Also I wonder if it's the Arrow community's role to standardize such > > > > extension types, or if we should let the Parquet community do it? > > > > > > > > Regards > > > > > > > > Antoine. > > > > > > > > > > > > > > > > Le 02/09/2026 à 22:51, Mandukhai Alimaa a écrit : > > > > > Hi everyone, > > > > > > > > > > Following the recent addition and subsequent refinement of the FILE > > > > > LogicalType to the Parquet format specification [1, 2] and the > > > > > corresponding Parquet C++ > > > > > reader/writer support currently underway [3], we (@rok and I) would > > > > > like to propose defining a matching Canonical Extension Type for > the > > > > > Apache Arrow specification: arrow.file. > > > > > > > > > > Motivation > > > > > As storage formats standardize file-level references, Arrow needs a > > > > > canonical in-memory representation to preserve these semantics > across > > > > > IPC streams, the C Data Interface, Arrow Flight, and language > > > > > bindings. Without a canonical extension type, file fields lose > their > > > > > semantic metadata during serialization. > > > > > > > > > > Proposed High-Level Design: > > > > > The intended design is to directly align with the finalized Parquet > > > spec > > > > [4]. > > > > > > > > > > * Extension name: ``arrow.file``. > > > > > * Storage Type: StructType containing standard optional fields: > > > > > * ``uri`` (Utf8) > > > > > * ``offset`` (Int64) > > > > > * `` size`` (Int64) > > > > > * ``content_type`` (Utf8) > > > > > * `` checksum`` (Utf8) > > > > > * ``inline`` (Binary) > > > > > > > > > > I have created a GitHub issue to track this [5]. Please share your > > > > > thoughts! Thanks! > > > > > > > > > > [1] https://github.com/apache/parquet-format/pull/585 > > > > > [2] https://github.com/apache/parquet-format/pull/603 > > > > > [3] https://github.com/apache/arrow/issues/50891 > > > > > [4] > > > > > > > > > > https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#file > > > > > [5] https://github.com/apache/arrow/issues/51143 > > > > > > > > > > > > > >
