Hello everyone, This is an issue I've had awareness of going way back but it's the first time I've had to personally deal with it. I've done some research and found it addressed in a number of ways but given current time pressures and a huge list of to-dos, I thought I'd turn to the community for rapid advice to get me over the hump. The circumstances are summarized below in code and comments suitable for a playground.
"This is the actual arithmetic I want to do... 0.09560268 - 0.005 = 0.09060268" "Evaluate this in a playground and you'll see this is what floating point arithmetic produces --the first two are wrong, every one after is correct" 0.09560268 - 0.005 "=> 0.09060267999999999". 0.0956026 - 0.005 "=> 0.09060259999999999". 0.095602 - 0.005 "=> 0.090602". 0.09560 - 0.005 "=> 0.0906". 0.0956 - 0.005 "=> 0.0906". 0.095 - 0.005 "=> 0.09". "This workaround works" 0.09560268 - 0.005 roundTo: 0.00000001 "=> 0.09060268". "but one has to compute number of digits of precision and according to study above, it's not necessary at precisions >= 0.000001" "What are other suggestions, workarounds or approaches community has?" "THANKS!" Don Howard <https://objectguild.com> *Founding Member* dhow...@objectguild.com +31653139744 (US) 651-253-7024
