It could also be reasonable to introduce additional metadata to SortField
to declare more details about the sort. Java and IEEE754 sort order are
both valid depending on context.
On Thu, Feb 27, 2025 at 6:00 PM Gang Wu wrote:
> FYI: there was an effort from Jan (cc'd) to introduce a total order
FYI: there was an effort from Jan (cc'd) to introduce a total order for
floating-point numbers on the Parquet side: [1][2].
[1] https://github.com/apache/parquet-format/pull/221
[2] https://github.com/apache/parquet-format/pull/196
On Thu, Feb 27, 2025 at 4:24 AM Devin Smith
wrote:
> The spec h
The spec https://iceberg.apache.org/spec/#sorting says
Sorting floating-point numbers should produce the following behavior: -NaN
> < -Infinity < -value < -0 < 0 < value < Infinity < NaN. This aligns with
> the implementation of Java floating-point types comparisons.
As far as I know, this does
