On Jun 18, 8:02 am, "Jerry Hill" <[EMAIL PROTECTED]> wrote: > On Wed, Jun 18, 2008 at 1:47 AM, <[EMAIL PROTECTED]> wrote: > >>>> 234 - 23234.2345 > > -23000.234499999999 > > > This is not correct by my calculations. > > Python floating point operations use the underlying C floating point > libraries which, in turn, usually rely on the hardware's floating > point implementations. This Wikipedia article talks about how those > values are usually stored and > manipulated:http://en.wikipedia.org/wiki/IEEE_floating-point_standard > > So, which IEEE double precision floating point value would you like > instead? As far as I understand it, these are your two choices: > > 'ba490c020f76d6c0' = -23000.234499999999 > 'bb490c020f76d6c0' = -23000.234500000002 > > Alternatively, investigate the Decimal module, but keep in mind that > it can have the same sorts of issues, just on different numbers. > Compare, for instance: > > >>> (1.0/3.0) * 3.0 > > 1.0 > > >>> from decimal import Decimal > >>> ( Decimal('1.0')/Decimal('3.0') ) * Decimal('3.0') > > Decimal("0.9999999999999999999999999999") > > > > -- > Jerry
So it seems then that python might not be very good for doing precision floating point work, because there is a good chance its floating points will be off by a (very small) amount? Or is there a way to get around this and be guaranteed an accurate answer? -- http://mail.python.org/mailman/listinfo/python-list
