Yes, you're right, that's why I'm rounding to 15 digits not 16. (or in the worst cases you can get it to 14 digits) Thanks for your input.
-----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Michael S. Davis Sent: Martes, 19 de Septiembre de 2000 5:22 pm To: Palm Developer Forum Subject: RE: Serious Palm FloatNumber problem I think you are missing something here. It is a fact that not ALL floating point numbers can be represented precisely in the 64 bits allowed. As a result of this, you will always have round of errors. Whenever you display a number in the same precision of the stored value, you will run the risk of exposing this error. And using the round() does not always correct the problem. Remember, the value shown is as exact as the palm can do. Using round() cannot give you more precision. For instance say you take .5 and subtract .4 and get .09999999999999999999 internally. How is round() supposed to know that you subtracted .4 from .5 and got that result (which was an error) vs two other numbers where the .099999999999999999 was actually correct. Whenever you try to display a number with exactly the same precision of the binary way it is stored, you have a potential round off problem. On Tue, 19 Sep 2000, Pablo Valle wrote: > OK, OK, I recognize that the error comes through because the exponent of (y) > is 0 thus the precision is 1E-16, but after all, the answer from > FlpCorrectedSub should be clean. > > I make my FlpCorrectedCorrectedSub > > FlpCompDouble FlpCorrectedCorrectedSub(FlpCompDouble x, FlpCompDouble y) > { > Int16 ex; > Int16 accuracy = 15; // 15 decimals to the right of the biggest > exponent > > ex = max(F64GetExp(x), F64GetExp(y)) ; // get the max exponent (the > F64GetExp is my func) > x.fd = FlpCorrectedSub(x.fd, y.fd, FlpAccuracy); // subtract the > two numbers > ex = accuracy - ex; > y.d = pow(10, ex); // fortunately I'm using MathLib (thanks > Rick Huebner) > x.d = x.d * y.d; > x.d = round(x.d); // clean the garbage > x.d = x.d / y.d; > return (x); // return a clean number > } > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of John > Marshall > Sent: Martes, 19 de Septiembre de 2000 11:36 am > To: Palm Developer Forum > Subject: Re: Serious Palm FloatNumber problem > > Pablo Valle <[EMAIL PROTECTED]> wrote: > > [...snip... 0.999999 - 1 ...] will give a result of > > -9.999999998067111E-7 > > ...and we all know that the answer is -0.0000010000000000 > > You do realise that > > |-9.999999998067111E-7 - -0.0000010000000000| = 1.932889E-16 < really > small > > and that that's about all the accuracy you should expect, right? > > You have read "What every computer scientist should know about floating > point", right? > > John "trolled again" > > -- > For information on using the Palm Developer Forums, or to unsubscribe, > please see http://www.palmos.com/dev/tech/support/forums/ > > > -- > For information on using the Palm Developer Forums, or to unsubscribe, please see http://www.palmos.com/dev/tech/support/forums/ > ---------------------------------------------------- Shoot-to-Win Protect the 2nd Amendment ---------------------------------------------------- -- For information on using the Palm Developer Forums, or to unsubscribe, please see http://www.palmos.com/dev/tech/support/forums/ -- For information on using the ACCESS Developer Forums, or to unsubscribe, please see http://www.access-company.com/developers/forums/
