Thanks Kosta --
For anyone else following along, my summary of the proposed change is to
explicitly disallow shreded columns in the Arrow representation
("typed_value") that have no equivalent type in Variant (e.g. unsigned
integers, or Null). It makes sense to me, but I do think it is important to
have a wider discussion before making any changes.
Andrew
On Tue, Aug 18, 2026 at 2:02 PM Kosta Tarasov <[email protected]>
wrote:
> Bumping up this thread.
>
> I submitted a PR to change the Arrow docs here -
> https://github.com/apache/arrow/pull/50810
>
> @alamb suggested that it should go through the mailing list first, because
> the changes can be considered as changing the spec.
>
> Please take a look and let me know what you think.
>
> Thanks,
> Kosta
>
>
>
> Sent with Proton Mail secure email.
>
> On Thursday, July 23rd, 2026 at 6:32 PM, Kosta Tarasov <
> [email protected]> wrote:
>
> > Hi everyone,
> >
> > I was reviewing a PR (https://github.com/apache/arrow-rs/pull/10417)
> for an issue in arrow-rs - https://github.com/apache/arrow-rs/issues/10416
> - and discovered an inconsistency between the Arrow spec and the Parquet
> spec.
> >
> > The issue proposed adding Unsigned integer typed_value support for
> Shredded Variant based on the Arrow spec -
> https://arrow.apache.org/docs/format/CanonicalExtensions.html#parquet-variant
> >
> > It says - "A field named typed_value which can be a Primitive Type
> Mappings or a List, LargeList, ListView or Struct". Where Primitive Type
> Mapping is
> https://arrow.apache.org/docs/format/CanonicalExtensions.html#primitive-type-mappings
> >
> > This table is inconsistent with the Shredded Value Types table in the
> Parquet Shredding Spec -
> https://github.com/apache/parquet-format/blob/master/VariantShredding.md#shredded-value-types
> >
> > Among other differences, the Arrow table allows Uint8/16/32 and Null
> typed_value columns, which have no valid Parquet shredded representation
> and can never round trip through Parquet. Implementations have already
> diverged over this: arrow-rs rejects unsigned typed_value, while Arrow Go
> accepts it and writes it to Parquet as INT(8/16/32, false), which the
> shredding spec does not allow.
> >
> > I propose refreshing the Arrow Parquet Variant spec table based on the
> one in the Parquet Shredding spec.
> >
> > I filed an issue in the Arrow repo with the proposed table -
> https://github.com/apache/arrow/issues/50622
> >
> > Thank you,
> > Kosta
>
