On Mon, Sep 30, 2019 at 10:30:59PM -0300, James Almer wrote: > On 9/30/2019 1:30 PM, Michael Niedermayer wrote: > > Fixes: -nan is outside the range of representable values of type 'unsigned > > short' > > From lrint documentation: > > "If x is a NaN or an infinity, or the rounded value is too large to be > stored in a long (long long in the case of the ll* functions), then a > domain error occurs, and the return value is unspecified." > > So i don't know if using lrint is a good idea here. > > > Fixes: > > 17769/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FITS_fuzzer-5678314672357376 > > Is the output of av_int2double/av_int2float or header.data_min NaN in > that testcase? Wouldn't it be better to check that instead, and abort?
If you prefer we can do that (check every pixel for being NaN and if not then within the range.) I was just hesitating a bit as this is likely going to result in speed loss And in the most strictest sense it is not preventing the error case as floats dont gurantee that level of consistency ... thanks [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Asymptotically faster algorithms should always be preferred if you have asymptotical amounts of data
signature.asc
Description: PGP signature
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
