On Tue, Jun 17, 2014 at 4:34 AM, J. Landman Gay <jac...@hyperactivesw.com> wrote: > set the numberformat to "0.##############" -- add more if you want
I think the reason David isn't doing that, and is trying to limit the number of # to the absolute minimum is because of this note in the Dictionary: Note: Since LiveCode does not use decimal numbers for its internal calculations (for reasons of speed), the decimal representation of a number is sometimes slightly off the correct number. For example, 10^-1 is equal to 0.1, but is calculated (to eighteen decimal places) as 0.100000000000000006. Because of this, setting the numberFormat to specify many decimal places after the decimal point may produce unexpected results in a statement that tests for an exact number. To prevent this, either avoid setting the numberFormat to a value more precise than you need, or use the abs function instead of the = operator to test equality: I think we all know that at least once or twice a year we get a post to this List questioning why LC is getting it's math wrong and the reason is as stated above. For anyone whose app requires precision math that shouldn't be a note, it should be a CAUTION! It's so easy to have your app work perfectly 99.9999999999999999% of the time... but then ruin someone's day. _______________________________________________ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
