On Sun, Sep 16, 2018 at 09:25:14PM +0200, Martin Vignali wrote:
> > you can just use something like (possibly with more or less 0 or a
> > magnitude
> > related value)
> > #define assert_stable_int(x) av_assert(llrintf(x+0.0001) ==
> > llrintf(x-0.0001))
> > #define assert_stable_float(x) av_assert((float)(x+0.000000000001) ==
> > (float)(x-0.000000000001))
> >
> > and then place this where rounding happens, then you can see easily if
> > any test gets close to problematic values. No need for special HW
> >
> > Hello,
>
> Do you mean something like that ? (which raise the assert for 8b and 16
> bits)
>
> #define assert_stable_float(x)
> av_assert0((float)(x+0.0000000000000000000000000001) ==
> (float)(x-0.0000000000000000000000000001))
>
> static void inline fill_uint_to_float_lut(SwsContext *c, int bitdepth) {
> static const double float_mult8 = 1.0 / 255.0;
> static const double float_mult16 = 1.0 / 65535.0;
> int i;
> double tmp;
>
> if (bitdepth == 8) { /*! fill uint8 to float lut */
> for (i = 0; i < 256; ++i){
> tmp = (double) i * float_mult8;
> assert_stable_float(tmp);
> c->uint2float_lut[i] = (float)tmp;
> }
> } else if (bitdepth == 16) { /*! fill uint16 to float lut */
> for (i = 0; i < 65536; ++i){
> tmp = (double) i * float_mult16;
> assert_stable_float(tmp);
> c->uint2float_lut[i] = (float)tmp;
> }
> } else { /*! unsupported bitdepth */
> av_assert0(0);
> }
> }yes, for which values exactly does it fail ? also, have you tried adding a small constant to tmp ? i would expect that this or a similar operation would allow moving away from all "unstable" points without really changing the output in a relevant way. thanks [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Many things microsoft did are stupid, but not doing something just because microsoft did it is even more stupid. If everything ms did were stupid they would be bankrupt already.
signature.asc
Description: PGP signature
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
