Re: round issue

2010-07-13 Thread Emile van Sebille
On 7/13/2010 9:06 AM Robin Becker said... Anyhow does anyone have a good algorithm for ensuring rounded percentages do add up to 100%? :) How about grouped percentages ie ensure the group sums and the groups display correctly in rounded form. I generally do the calculations in memory then set

Re: round issue

2010-07-13 Thread Robin Becker
On 12/07/2010 19:59, Mark Dickinson wrote: It does look inconsistent however, and it seems to me rounding and interpolation should behave similarly. Agreed. In this case it's a minor bug that round(-9.85, 1) produces -9.9 instead of -9.8; both string formatting and round should g

Re: round issue

2010-07-12 Thread Mark Dickinson
Emile van Sebille fenx.com> writes: > > On 7/12/2010 2:52 AM Robin Becker said... > > > What value should round(-9.85,1) return? > > Per round's definition, -9.9. No. The float that's represented by the literal '-9.85' *isn't* exactly -9.85, for all the usual binary floating-point reasons. T

Re: round issue

2010-07-12 Thread Emile van Sebille
On 7/12/2010 2:52 AM Robin Becker said... What value should round(-9.85,1) return? Per round's definition, -9.9. String interpolation for %n.mf doesn't appear to define it's rounding behavior, so a peek at the source would answer what's being done. It does look inconsistent however, and i

Re: round issue

2010-07-12 Thread Gary Herron
On 07/12/2010 02:52 AM, Robin Becker wrote: A client wants to know why his db number -9.85 gets displayed by some simple code as -9.8 I looked at the number and see that >>> -9.85 -9.8496 ie I expect simple rounding to produce the observed result and indeed >>> '%.1f' % -9.85 '-9

Re: round issue

2010-07-12 Thread Mark Dickinson
On Jul 12, 10:52 am, Robin Becker wrote: > What value should round(-9.85,1) return? Is the result explainable in python > (ie > without resort to the internal FP representations etc etc)? As you observe, the closest float to -9.85 is actually just a little smaller (i.e., closer to 0) than -9.85:

round issue

2010-07-12 Thread Robin Becker
A client wants to know why his db number -9.85 gets displayed by some simple code as -9.8 I looked at the number and see that >>> -9.85 -9.8496 ie I expect simple rounding to produce the observed result and indeed >>> '%.1f' % -9.85 '-9.8' however, when I use round I get an unexp