You say that the first two results are wrong. No. The first two results are *exactly* what they are supposed to be. Floating point arithmetic is NOT mathematical real arithmetic. The floating point *numbers* are exact rational numbers of the form n * 2^k where n is a (bounded) integer and k is a (bounded) integer. If you enter a number that is not of this form (and within bounds) you will get a *different* number that *is* of that form and is close. The floating point *operations* in IEEE arithmetic are exact when and only when the result is a number of that form (and within bounds), otherwise the result is *approximate*.
If you want exact arithmetic on numbers of the form n * 10^k, ANSI Smalltalk provides the ScaledDecimal class, which, sigh, Squeak and Pharo interpret as "rational number *printed* in decimal form but not *computed* using decimal arithmetic", so that can give extremely confusing results too. So what you are going to have to do is use rational numbers, which are in Smalltalk the union of Integer and Fraction. On Mon, 13 Jan 2020 at 09:13, Donald Howard <dhow...@objectguild.com> wrote: > Hello everyone, > > This is an issue I've had awareness of going way back but it's the first > time I've had to personally deal with it. I've done some research and > found it addressed in a number of ways but given current time pressures and > a huge list of to-dos, I thought I'd turn to the community for rapid advice > to get me over the hump. The circumstances are summarized below in code > and comments suitable for a playground. > > "This is the actual arithmetic I want to do... > 0.09560268 - 0.005 = 0.09060268" > > "Evaluate this in a playground and you'll see this is what > floating point arithmetic produces --the first two are wrong, every one > after is correct" > 0.09560268 - 0.005 "=> 0.09060267999999999". > 0.0956026 - 0.005 "=> 0.09060259999999999". > 0.095602 - 0.005 "=> 0.090602". > 0.09560 - 0.005 "=> 0.0906". > 0.0956 - 0.005 "=> 0.0906". > 0.095 - 0.005 "=> 0.09". > > "This workaround works" > 0.09560268 - 0.005 roundTo: 0.00000001 "=> 0.09060268". > "but one has to compute number of digits of precision > and according to study above, it's not necessary at > precisions >= 0.000001" > > "What are other suggestions, workarounds or approaches community has?" > "THANKS!" > > > Don Howard > > <https://objectguild.com> > *Founding Member* > dhow...@objectguild.com > +31653139744 > (US) 651-253-7024 >
