> > On 8 August 2011 03:58, Jose Fonseca <jfons...@vmware.com> wrote: > > > It's subjective. It depends on the expected input distribution, > > > which is effectively impossible to characterize in general. One > > > can easily find datasets where one method gives biased results and > > > the other not, and vice versa. And if one takes all possible > > > numbers, they are equally good. > > > > > This is probably largely irrelevant to the patch in question, but > > just > > for arguments sake, I don't think that's true. The function floor(x + > > .5) will introduce positive bias regardless of input distribution, > > I thought the patch implemented round to +/- infinity. Indeed round to > +infinitity indeed has a bias.
IMHO, positive numbers are more common in shaders than negative, and for those, the floor(x+0.5) approach would create correct results (= human rounding, away from zero) while rndne would not. Will rewrite to use rndne though if that's the consensus. - Lauri _______________________________________________ mesa-dev mailing list mesa-dev@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/mesa-dev
