On 10.08.2022 23:24, Andreas Rheinhardt wrote:
Timo Rothenpieler:
IEEE-754 differentiates two different kind of NaNs.
Quiet and Signaling ones. They are differentiated by the MSB of the
mantissa.
For whatever reason, actual hardware conversion of half to single always
sets the signaling bit to 1 if the mantissa is != 0, and to 0 if it's 0.
So our code has to follow suite or fate-testing hardware float16 will be
impossible.
What does the exr spec say about quiet and signaling nans?
Not sure how exr would be involved here.
But I tested this on both aarch64, x86 with sse2 emulation and x86 f16c
on alderlake and zen2.
They all perfectly agree and match 100% what this changed code produces
for the entire range of 65k possible values.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
