> > also there are 2 divisions in this that you can trivially eliminate > /255 and /65535 (extra precission beyond IEEE float/double could change > these) > > also the whole could be done with fewer floats and no extra complexity > for example: > int64_t tmp2 = 16843009LL * i; > (float)((double)tmp2 / (1LL<<32)) > and > int64_t tmp2 = 4295032833LL * i or uint64_t 281479271743489 > > Still raise the assert for me. Maybe we can go back to the initial patch i propose. Let the simple initial code, for non bitexact conversion and use a dedicated calc for bit exact version, without float calc
Martin _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
