Re: rounding problem

2005-02-25 Thread Nick Coghlan
Dan Bishop wrote: Your statement is misleading, because it suggests that your processor stores digits. It doesn't; it stores *bits*. And where does the word 'bit' come from, hmm? It couldn't possibly be an abbreviation of Binary digIT, could it? Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECT

Re: rounding problem

2005-02-24 Thread Thomas Bartkus
"Dan Bishop" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > tom wrote: > > That last digit will *always* contain some arithmetic slop. > > Your statement is misleading, because it suggests that your processor > stores digits. It doesn't; it stores *bits*. Your explanation is much

Re: rounding problem

2005-02-23 Thread Dan Bishop
tom wrote: > On Wed, 23 Feb 2005 19:04:47 -0600, Andy Leszczynski wrote: > > > It is on Windows, Linux, Python 2.3: > > > > [GCC 3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)] on linux2 Type "help", > > "copyright", "credits" or "license" for more information. > > >>> a=1.1 > > >>> a > > 1.1

Re: rounding problem

2005-02-23 Thread tom
On Wed, 23 Feb 2005 19:04:47 -0600, Andy Leszczynski wrote: > It is on Windows, Linux, Python 2.3: > > [GCC 3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)] on linux2 Type "help", > "copyright", "credits" or "license" for more information. > >>> a=1.1 > >>> a > 1.1001 > >>> > >>> > >>> >

Re: rounding problem

2005-02-23 Thread Michael Hartl
> Is it normal? Yes. The interpreter prints back the repr of a, which reflects the imprecision inherent in floats. If you want '1.1', use the string returned by the str function. >>> a = 1.1 >>> a 1.1001 >>> repr(a) '1.1001' >>> str(a) '1.1' Michael -- Michael D. Hartl

Re: rounding problem

2005-02-23 Thread Kristian M Zoerhoff
In article <[EMAIL PROTECTED]>, Andy Leszczynski wrote: > >>> a=1.1 > >>> a > 1.1001 > >>> > > > Is it normal? Yes, for floating-point numbers. This is due to inherent imprecision in how floats are represented in hardware. If you can live with being a touch off that many decimal plac

Re: rounding problem

2005-02-23 Thread Aahz
In article <[EMAIL PROTECTED]>, Andy Leszczynski <[EMAIL PROTECTED]> wrote: >It is on Windows, Linux, Python 2.3: > >[GCC 3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)] on linux2 >Type "help", "copyright", "credits" or "license" for more information. > >>> a=1.1 > >>> a >1.1001 > >>> > > >Is