On Thu, Mar 03, 2011 at 06:23:36PM -0800, Folkes, Michael wrote: > Perhaps somebody could clarify for me if the following is a floating > point matter or otherwise, and how am I to correct for it? > > > floor(100*.1) > [1] 10 > > > 100*(1.0-.9) > [1] 10 > > > floor(100*(1-0.9)) > [1] 9
As others pointed out, 0.1 is not exactly representable in base 2, so we get formatC(0.1, digits=20) [1] "0.10000000000000000555" formatC(100*0.1, digits=20, width=-1) [1] "10" formatC(100*(1 - 0.9), digits=20) [1] "9.9999999999999982236" A correct result may be obtained, if you reorganize your calculation, so that all intermediate results are integers and the inaccurate division is only the last operation. Then, floor(n/10) will be correct and also n %/% 10 may be used. Alternatively, if you work with decimal numbers with 1 or 2 decimal digits, then also floor(round(x, 1)) or floor(round(x, 2)) work correctly, if x is not too large. See FAQ 7.31 http://cran.r-project.org/doc/FAQ/R-FAQ.html#Why-doesn_0027t-R-think-these-numbers-are-equal_003f and http://rwiki.sciviews.org/doku.php?id=misc:r_accuracy:decimal_numbers for further examples and some hints. Hope this helps. Petr Savicky. ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
