On Mon, Sep 10, 2018 at 07:57:42PM +0200, Martin Vignali wrote: > > > > does the LUT generation code produce different results on platforms ? > > if so i would suggest to try to use double and to add a small offset if > > needed > > > > a 8bit table has 256 entries, a 16bit table 65536 > > a difference would occur if a source value from 64bit floats gets rounded > > differently to 32bit floats. If this occurs a small offset could be added > > so that none of the 65536 cases end up close to being between 2 32bit > > floats > > > > This would avoid teh rather complex code if it works > > > > > Hello, > > Can't test on other platform than x86_32 and x86_64, so i can't really > answer to this question.
> It's the reason why, i write a code, which doesn't use float calc. you can just use something like (possibly with more or less 0 or a magnitude related value) #define assert_stable_int(x) av_assert(llrintf(x+0.0001) == llrintf(x-0.0001)) #define assert_stable_float(x) av_assert((float)(x+0.000000000001) == (float)(x-0.000000000001)) and then place this where rounding happens, then you can see easily if any test gets close to problematic values. No need for special HW [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Let us carefully observe those good qualities wherein our enemies excel us and endeavor to excel them, by avoiding what is faulty, and imitating what is excellent in them. -- Plutarch
signature.asc
Description: PGP signature
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
