On 10.08.2022 23:43, Andreas Rheinhardt wrote:
Timo Rothenpieler:
On 10.08.2022 23:24, Andreas Rheinhardt wrote:
Timo Rothenpieler:
IEEE-754 differentiates two different kind of NaNs.
Quiet and Signaling ones. They are differentiated by the MSB of the
mantissa.
For whatever reason, actual hardware conversion of half to single always
sets the signaling bit to 1 if the mantissa is != 0, and to 0 if it's 0.
So our code has to follow suite or fate-testing hardware float16 will be
impossible.
What does the exr spec say about quiet and signaling nans?
Not sure how exr would be involved here.
Your patch changes the output of an exr-test. The output of the exr
decoder is presumably determined by the exr spec. There is after all the
possibility that what hardware does in hardware and what this patch does
in software is incompatible with what exr specifies.
The exr spec just says something along the lines of analogous to
ieee-754 floats:
https://openexr.readthedocs.io/en/latest/TechnicalIntroduction.html?highlight=ieee#the-half-data-type
It barely ever mentions NaNs, other than that they exist. Which makes
sense, given they don't typically appear in images.
The only output changed is that for how NaNs are converted.
And given the cross-validation with multiple hardware implementations,
I'm confident that it's correct.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
