Thanks, everyone, for the great input. I will continue to work thru the
background readings and recommendations you graciously supplied and go from
there.
Best regards,
- Don
On Tue, 14 Jan 2020 at 07:04, Bruce O'Neel wrote:
> HI,
>
> I would upgrade that "a good read" to a critical read if yo
HI,
I would upgrade that "a good read" to a critical read if you do anything with
floating point. Floating point numbers are not real numbers but rather a small
subset, and, have all kinds of funny properties. But we use them just like real
numbers and that's what catches us out.
che
On Tue, Jan 14, 2020 at 12:41:51AM +1300, Richard O'Keefe wrote:
> Squeak and Pharo interpret as "rational number *printed* in
> decimal form but not *computed* using decimal arithmetic", so
> that can give extremely confusing results too.
Can you expand on this please?
For money-related calculat
On Sun, Jan 12, 2020 at 02:13:00PM -0600, Donald Howard wrote:
> "What are other suggestions, workarounds or approaches community has?"
This is a good read on the subject:
https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
Title is "What every computer scientist should know about
Ah, yes, yes, sending #asScaledDecimal to a Float is too late, the precision is
already lost. It makes total sense now.
Thank you, Werner, for the explanation.
> On 13 Jan 2020, at 16:13, werner kassens wrote:
>
> Hi Sven,
>
> if you dont transform floats, that are not always exactly what the
Hi Sven,
if you dont transform floats, that are not always exactly what they seem
to be, you get what you would expect:
0.09560268s8 - 0.005s8 = 0.09060268s8. "true"
"of course"
a:=0.09560268 asScaledDecimal.
0.09560268s8=a . "false"
"you could eg do this:"
b:=a roundTo: 0.0001s8.
"then"
0.095
> On 13 Jan 2020, at 00:28, Paul DeBruicker wrote:
>
> Whats wrong with #roundedTo: ?
>
>
> If you actually need it calculated to 8 significant digits then you can use
> scaled decimals e.g.
>
> (0.09560268 asScaledDecimal - 0.005 asScaledDecimal)
The weird thing is though, that is still
On Mon, 13 Jan 2020 at 19:42, Richard O'Keefe wrote:
> You say that the first two results are wrong.
> No. The first two results are *exactly* what they are supposed to be.
> Floating point arithmetic is NOT mathematical real arithmetic.
> The floating point *numbers* are exact rational numbers
You say that the first two results are wrong.
No. The first two results are *exactly* what they are supposed to be.
Floating point arithmetic is NOT mathematical real arithmetic.
The floating point *numbers* are exact rational numbers of the form
n * 2^k where n is a (bounded) integer and k is a (
On 12/01/2020 21:13, Donald Howard wrote:
"Evaluate this in a playground and you'll see this is what
floating point arithmetic produces --the first two are wrong, every
one after is correct"
0.09560268 - 0.005 "=> 0.090602679".
0.0956026 - 0.005 "=> 0.090602599".
0.095602 - 0.
Whats wrong with #roundedTo: ?
If you actually need it calculated to 8 significant digits then you can use
scaled decimals e.g.
(0.09560268 asScaledDecimal - 0.005 asScaledDecimal)
see https://0.30004.com for more about why floating point math
does this sometimes
Donald Howa
Hello everyone,
This is an issue I've had awareness of going way back but it's the first
time I've had to personally deal with it. I've done some research and
found it addressed in a number of ways but given current time pressures and
a huge list of to-dos, I thought I'd turn to the community for
12 matches
Mail list logo
