On Fri, Jun 05, 2015 at 09:27:56AM +0000, Nedeljko Babic wrote:
> >> >> - while((a.mant + 0x20000000U)<0x40000000U){
> >> >> + while((FFABS(a.mant) + 0x20000000U)<0x40000000U){
> >> >
> >> >when exactly is this needed and how does a.mant reach that value
> >> >for which this is needed?
> >> >FFABS would significantly slow this down i suspect and it just looks
> >> >wrong
> >> >
> >>
> >> The problem occurs for some negative numbers during normalization.
> >>
> >> For example, please consider if 0xFFFFFFD80 is sent to av_normalize_sf.
> >
> >considering that this is not a 32bit value and mant is, theres no
> >way that can happen
> >
> >
> >>
> >> The while loop has a condition: (a.mant + 0x20000000U)<0x40000000U.
> >> 0xFFFFD80 is cast to unsigned and we have:
> >
> >and that lost its highest 8bit
> >also FFABS() does not chaneg the result for this
>
> Ok, this was relay bad example (and explanation)... Sorry.
>
> Lets try another example:
>
> int32_t a = 0xFFFFFF00;
> SoftFloat sf = av_int2sf(a, xxx); // xxx is not important here
>
> If original condition in normalization is used we will have:
> sf.mant = 0xc0000000
>
> If FFABS is used in condition, there will be one iteration less and we will
> have:
> sf.mant = 0xe0000000fixed thanks [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Opposition brings concord. Out of discord comes the fairest harmony. -- Heraclitus
signature.asc
Description: Digital signature
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
