Re: [FFmpeg-devel] [PATCH v2 2/7] avutil/half2float: adjust conversion of NaN

2022-08-18 Thread Tomas Härdin
sön 2022-08-14 klockan 18:48 +0200 skrev Timo Rothenpieler: > IEEE-754 differentiates two different kind of NaNs. > Quiet and Signaling ones. They are differentiated by the MSB of the > mantissa. > > For whatever reason, actual hardware conversion of half to single > always > sets the signaling bi

[FFmpeg-devel] [PATCH v2 2/7] avutil/half2float: adjust conversion of NaN

2022-08-14 Thread Timo Rothenpieler
IEEE-754 differentiates two different kind of NaNs. Quiet and Signaling ones. They are differentiated by the MSB of the mantissa. For whatever reason, actual hardware conversion of half to single always sets the signaling bit to 1 if the mantissa is != 0, and to 0 if it's 0. So our code has to fol