Re: Clarification on DayTransform Result Type

Thanks for confirming! To close the loop on this issue, we have added more documentation about the `result_type` function in PyIceberg. This clarifies the physical and display representations of partition transforms. For DayTransform, the physical representation is `int`, while the display represe

Re: Clarification on DayTransform Result Type

Yes. When we return the Spark type, it shows up as date and Spark correctly displays the value. On Mon, Sep 30, 2024 at 9:56 AM Kevin Liu wrote: > Thank you both for the insights and context. > > As Russell pointed out, the "day partition transform" result is true of > int type. The Types.DateTy

Re: Clarification on DayTransform Result Type

Thank you both for the insights and context. As Russell pointed out, the "day partition transform" result is true of int type. The Types.DateType correspo

Re: Clarification on DayTransform Result Type

The background is that the result of the day function and dates are basically the same: the number of days from the Unix epoch. When we started using metadata tables, we realized that a lot of people use the day function but then get a weird ordinal value out, but if we just change the type to `dat

Re: Clarification on DayTransform Result Type

Good thing DateType is an Integer :) https://github.com/apache/iceberg/blob/113c6e7d62e53d3e3cb15b1712f3a1db473ca940/api/src/main/java/org/apache/iceberg/types/Type.java#L37 On Thu, Sep 26, 2024 at 8:38 PM Kevin Liu wrote: > Hey folks, > > While reviewing a PR to fix DayTransform in PyIceberg (#

Clarification on DayTransform Result Type

Hey folks, While reviewing a PR to fix DayTransform in PyIceberg (#1208 ), we found an inconsistency between the spec and the Java Iceberg library. According to the spec , the result type fo